-- Logs begin at Wed 2022-08-24 07:00:07 UTC, end at Wed 2022-08-24 08:39:06 UTC. -- Aug 24 07:31:09.153995 dv-cinder-854274-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 07:31:09.162412 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 07:31:09.164773 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 07:31:09 2022] *** Aug 24 07:31:09.164773 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 07:31:09.164773 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 07:31:09.164773 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: nodename: dv-cinder-854274-1-iscsi Aug 24 07:31:09.164773 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: machine: x86_64 Aug 24 07:31:09.164773 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: clock source: unix Aug 24 07:31:09.164773 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: pcre jit disabled Aug 24 07:31:09.164773 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: detected number of CPU cores: 4 Aug 24 07:31:09.164773 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: current working directory: / Aug 24 07:31:09.164773 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: detected binary path: /usr/bin/uwsgi-core Aug 24 07:31:09.164773 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: your processes number limit is 47680 Aug 24 07:31:09.164773 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: your memory page size is 4096 bytes Aug 24 07:31:09.164773 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: detected max file descriptor number: 2048 Aug 24 07:31:09.164773 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: lock engine: pthread robust mutexes Aug 24 07:31:09.165575 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: thunder lock: enabled Aug 24 07:31:09.165575 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 07:31:09.165575 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 07:31:09.204052 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: Python main interpreter initialized at 0x56076a5209c0 Aug 24 07:31:09.204052 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: python threads support enabled Aug 24 07:31:09.204052 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: your server socket listen backlog is limited to 100 connections Aug 24 07:31:09.204052 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: your mercy for graceful operations on workers is 90 seconds Aug 24 07:31:09.204367 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 07:31:09.204467 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: *** Operational MODE: preforking *** Aug 24 07:31:09.204756 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: *** uWSGI is running in multiple interpreter mode *** Aug 24 07:31:09.204756 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: spawned uWSGI master process (pid: 88243) Aug 24 07:31:09.204947 dv-cinder-854274-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 07:31:09.212865 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: spawned uWSGI worker 1 (pid: 88249, cores: 1) Aug 24 07:31:09.212865 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: spawned uWSGI worker 2 (pid: 88250, cores: 1) Aug 24 07:31:09.212865 dv-cinder-854274-1-iscsi devstack@keystone.service[88243]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 07:31:13.608350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 07:31:13.608350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: from cryptography.utils import int_from_bytes Aug 24 07:31:13.609142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 07:31:13.609142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: from cryptography.utils import int_from_bytes Aug 24 07:31:13.612269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 07:31:13.612269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: from cryptography.utils import int_from_bytes Aug 24 07:31:13.612755 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 07:31:13.612755 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: from cryptography.utils import int_from_bytes Aug 24 07:31:13.660661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 07:31:13.660837 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 07:31:13.660927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 07:31:13.661018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 07:31:13.661098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] command line args: [] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 07:31:13.661180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] command line args: [] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 07:31:13.661301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 07:31:13.661388 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 07:31:13.661465 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ================================================================================ {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 07:31:13.661542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ================================================================================ {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 07:31:13.661721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] admin_token = **** {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.661812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] admin_token = **** {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.661965 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] config_dir = [] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.662051 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] config_dir = [] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.662155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.662240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.662341 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] config_source = [] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.662430 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] config_source = [] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.662503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] debug = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.662578 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] debug = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.662708 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.662977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.663133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.663218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.663279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.663516 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.663641 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.663791 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.663918 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.664046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.664157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] list_limit = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.664259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] log_config_append = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.664359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] list_limit = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.664465 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.664566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] log_config_append = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.664692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] log_dir = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.664794 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.664900 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] log_file = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.664996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] log_dir = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.664996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] log_file = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.665180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.665289 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.665372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.665444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.665608 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.665700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.665816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.665816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.666003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.666085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.666166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.666289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.666365 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.666479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.666553 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.666666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.666775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.666883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.667010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.667114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.667243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.667345 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.667473 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.667592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.667686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.667838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.667916 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.668109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.668197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.668412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.668630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.668835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] publish_errors = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.669005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.669085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.669206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] publish_errors = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.669348 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.669424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.669494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.669634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.669712 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.669831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.669924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.670050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.670126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] standard_threads = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.670277 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.670356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.670514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] standard_threads = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.670597 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.670736 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.670813 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.670912 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] use_journal = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.670990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.671072 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] use_json = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.671144 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.671247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] use_stderr = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.671333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] use_journal = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.671410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] use_syslog = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.671482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] use_json = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.671625 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] use_stderr = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.671703 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.671799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] use_syslog = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.671883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.672030 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:31:13.672108 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.672287 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.672368 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.672526 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.672611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.672839 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.672918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.673076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.673163 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.673323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.673407 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.673560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.673638 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.673804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.673882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.674038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.674116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.674286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.674365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.674518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.674601 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.674761 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.674845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.674992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.675069 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.675236 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.675324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.675475 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.675558 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.675708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.675793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.675954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.676037 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.676191 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.676269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.676433 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.676521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.676684 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.676775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.676938 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.677022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.677197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.677286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.677448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.677526 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.677696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.677780 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.677926 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.678078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.678152 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.678223 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.678374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.678452 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.678707 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.678796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.678916 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.679134 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.679212 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.679380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.679458 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.679609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.679688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.679855 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.679940 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.680101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.680219 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.680328 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.680409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.680566 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.680664 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.680820 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.680898 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.681054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.681166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.681289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.681399 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.681524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.681642 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.681752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.681850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.681971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.682092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.682198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.682285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.682419 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.682499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.682660 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.682783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.682872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.682960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.683107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.683186 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.683333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.683412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.683565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.683653 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.683797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.683877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.684028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.684109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.684258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.684336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.684492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.684589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.684745 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.684827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.685000 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.685088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.685214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.685307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.685448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.685533 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.685674 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.685756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.685907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.685986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.686138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.686217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.686367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.686444 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.686594 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.686678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.686841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.686919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.687073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.687151 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.687302 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.687394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.687533 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.687613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.687765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.687844 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.688001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.688087 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.688240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.688320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.688497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.688610 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.688740 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.688833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.688978 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.689057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.689211 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.689335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.689445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.689600 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.689676 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.689748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.689914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.689998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.690146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.690237 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.690388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.690468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.690623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.690701 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.690858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.690950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.691091 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.691172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.691326 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.691405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.691562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.691641 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.691809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.691885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.692047 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.692129 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.692288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.692367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.692526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.692606 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.692792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.692872 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.693030 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.693109 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.693268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.693348 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.693503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.693588 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.693734 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.693819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.693966 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.694049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.694746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.694831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.695061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.695140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.695296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.695538 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.695617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.695772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.695850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.696020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.696110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.696255 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.696333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.696497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.696580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.696761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.696856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.697012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.697096 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.697245 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] auth.external = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.697324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.697477 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.697564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.697747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.697826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.697991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.698070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.698220 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] auth.password = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.698328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] auth.external = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.698450 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.698538 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] auth.token = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.698690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.698770 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.698927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.699006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.699173 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.699260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] auth.password = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.699404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.699491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] auth.token = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.699641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.699720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.699878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.699966 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.700108 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.700187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.700346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.700425 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] credential.caching = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.700588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.700721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.700860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.700940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.701106 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.701223 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.701374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.701506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] credential.caching = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.701640 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.701769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.701898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.702023 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.702160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.702287 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.702412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.702535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.702675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.702802 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.702942 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.703070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.703153 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.703324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.703456 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.703583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.703583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.703830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.703956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.704082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.704082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.704331 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.704526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] federation.caching = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.704672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.704798 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.704926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.705065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.705192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.705325 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.705410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.705485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.705558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] federation.caching = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.705631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.705704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.705788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.705867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.705940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.706012 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.706085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.706157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.706263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.706345 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.706501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.706586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.706749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.706828 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.706983 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity.caching = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.707064 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.707284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.707373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.707535 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.707615 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.707772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.707853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.708007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity.caching = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.708086 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.708254 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.708333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.708486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.708567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.708744 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.708835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.708981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.709062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.709227 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.709308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.709457 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.709541 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.709688 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.709795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.710476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.710563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.710724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.710985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.711066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.711279 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.711390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.711515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.711649 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.711751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.711898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.711985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.712174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.712253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.712411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.712491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.712698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.712795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.712902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.713105 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.713197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.713271 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.713344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.713516 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.713644 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.713755 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.713849 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.713989 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.714129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.714215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.714291 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.714442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.714523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.714677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.714757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.714914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.715001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.715153 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.715233 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.715397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.715527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.715664 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.715790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.715917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.716046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.716174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.716312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.716446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.716671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.716797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.716973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.717140 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.717273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.717410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.717543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.717671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.717796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.717922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.718049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.718181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.718320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.718446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.718571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.718697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.718822 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.718955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.719081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.719245 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.719372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.719455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.719561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.719651 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.719727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.719799 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.719871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.719948 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.720021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.720101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.720174 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.720246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.720318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.720390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.720470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.720553 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.720626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.720743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.720824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.720991 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.721070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.722924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.723020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.723020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.723161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.723236 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.723310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.723393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.723469 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.723542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.723879 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.723952 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.724024 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.724106 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.724180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.724249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.724357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.724430 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.724502 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.724583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.724684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.724760 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.724832 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.724905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.724978 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.725065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.725141 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.725213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.725315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.725439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.725512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.725849 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.725913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.725974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.726034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.726101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.726177 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.726241 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.726301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.726361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.726361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.726466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.726534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.726596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.726656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.726716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.726867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.726945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.727104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.727182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.727339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.727417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.727576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.727653 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.727822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.728041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.728041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.728287 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.728414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.728539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.728724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.728852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.728980 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.729106 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] resource.caching = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.729233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.729358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.729492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.729617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.729760 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.729889 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.730017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.730144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.730279 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.730406 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.730406 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.730643 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] resource.caching = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.730768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.730894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.731028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.731155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.731281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.731417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.731543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] role.caching = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.731669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.731810 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] role.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.731933 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.732059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.732185 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.732310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.732436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.732568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.732744 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.732873 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.732993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.732993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] role.caching = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.733241 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.733379 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] role.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.733504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.733629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.733753 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.733878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.734003 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.734137 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.734270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.734352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.734518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.734643 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.734768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.734911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.735037 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.735164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.735288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.735288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.735526 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.735526 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.735792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.735918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.736044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.736176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.736310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.736437 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.736574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.736720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.736842 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.736968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.737108 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.737108 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.737346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.737472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.737599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.737735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.737874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.737959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.738089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.738221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.738348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.738474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.738609 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.738734 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.738860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.738985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.739110 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.739234 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.739365 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.739490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.739615 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.739741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.739864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.739999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.740131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.740256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.740380 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.740505 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] token.caching = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.740629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.740798 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.740934 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.741080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.741201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.741327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.741481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.741735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.741884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.742009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.742135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.742260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.742342 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] token.caching = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.742405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.742525 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.742604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.742677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.742750 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.742750 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.742882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.742963 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.743037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.743110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.743181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.743252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.743323 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.743402 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.743476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.743548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.743620 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.743698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.743771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.743851 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.743925 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.743997 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.connection = **** {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.744077 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.744158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.744232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.744304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.744384 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.744458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.744530 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.744602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.744689 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.744763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.745829 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.745944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.746065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.746142 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.746214 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.connection = **** {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.746295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.746377 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.746460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.746534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.746607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.746679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.746751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.746751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.746954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.747031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.747104 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.747176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.747248 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.747321 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.747401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.747475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.747547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.747619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.747691 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.747763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.747843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.747916 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 07:31:13.747989 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.747989 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.748194 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.748430 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.748695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.748930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.749169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.749404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:31:13.749574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 07:31:13.777134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.777359 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.777579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.778640 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.778872 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.779096 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.789720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.789884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.817248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.817473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.818156 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.818388 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.872436 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.872753 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.923812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.924044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.925211 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:13.925444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.086314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.086554 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.086640 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.086780 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.086894 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.087061 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.087149 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.087214 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.087276 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.087418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.087497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.087684 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.112162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.112385 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.112506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.112590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.112697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.112817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.112901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.113031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.113133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.113242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.113324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.113452 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.113533 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.113665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.125253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.125619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.138374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.138500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.138590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.138702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.138811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.138928 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.139034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.139161 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.139298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.139388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.139469 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.139583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.139678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.139802 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:31:14.157146 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.157357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 07:31:14.157695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.157778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.157914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 07:31:14.157984 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 07:31:14.158155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.158269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.158417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.158500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 07:31:14.158744 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.158930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.159014 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.159193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.159536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.159805 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.168867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.169149 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.169279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.169440 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.169567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.169706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.169913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.170191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.176667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.177325 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.180368 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.180741 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.180875 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.180993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.186357 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.186456 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.186534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.186609 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.186683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.186753 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.186816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.186886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.186953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.187015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.187076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.187136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.187205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.187269 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.187269 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.187378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.187378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.187378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.206069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.206748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.207515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.207800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.208218 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.208510 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.212757 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.213043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.213297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.213380 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.218999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.218999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.219643 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.219931 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.220412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.220729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.224276 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.224569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.224876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.225151 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.225236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.225960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.226054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.226253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.226373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.226563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.226693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.226849 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.226968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.227133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.227410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.227683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.244318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.244596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.245019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.245318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.248464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.249342 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.251665 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.251976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.252606 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.252938 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.256184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.257825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.260186 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.261007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.263516 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.263818 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.264104 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.264326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.264408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.264632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.266550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.266681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.266760 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.266834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.266834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.266834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.281310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.281594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.281872 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.282181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.282628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.282924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.283211 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.283483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.289895 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.291026 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.291400 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.291680 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.292548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.292855 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.295245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.295245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.295245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.295245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.295629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.295704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.295903 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.296223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.296522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.296837 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.307101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.307385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.307669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.307952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.308235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.308510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.308599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.308811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.308894 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.309171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.309454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.309745 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.310029 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.310311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.325020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.326766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.328024 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.329849 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.330983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.332411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.332795 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.333884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.334177 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.334259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.334468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.335804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.336099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.336441 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.341242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.342687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.343093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.344663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.345580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.345882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.346167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.346454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.347607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.347914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.348207 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.348499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.355467 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.355759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.356044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.357354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.357656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.357946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.362964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.363245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.363532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.363814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.364089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.364372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.364674 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.364929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:31:14.365013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.365221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.365305 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.365489 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.365579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.365768 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.365849 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.366623 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.366750 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.367137 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.367227 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.367436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.367610 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.367906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.368199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.368481 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.368802 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.369081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.369375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.369645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:31:14.402349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56076a5209c0 pid: 88249 (default app) Aug 24 07:31:14.404603 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56076a5209c0 pid: 88250 (default app) Aug 24 07:31:14.405822 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180ad56-d706-4aa9-ba3d-8abdde9bee4a None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:14.406109 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180ad56-d706-4aa9-ba3d-8abdde9bee4a None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:14.406383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180ad56-d706-4aa9-ba3d-8abdde9bee4a None None] PATH_INFO: `/v3` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:14.410528 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Wed Aug 24 07:31:14 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:21.863284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c893c5-6086-4eef-beb5-45ddc6bc5db7 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:21.863799 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c893c5-6086-4eef-beb5-45ddc6bc5db7 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:21.863880 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c893c5-6086-4eef-beb5-45ddc6bc5db7 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:21.868823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:21.875251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4d6d5-a107-489e-80fa-957ecb04f911 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:21.875656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4d6d5-a107-489e-80fa-957ecb04f911 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:21.875801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4d6d5-a107-489e-80fa-957ecb04f911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:21.945507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: INFO dbcounter [None req-54a4d6d5-a107-489e-80fa-957ecb04f911 None None] Registered counter for database keystone Aug 24 07:31:21.966314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG oslo_db.sqlalchemy.engines [None req-54a4d6d5-a107-489e-80fa-957ecb04f911 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88249) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 07:31:21.974197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writer thread running {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 07:31:22.152081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG passlib.handlers.bcrypt [None req-54a4d6d5-a107-489e-80fa-957ecb04f911 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=88249) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 07:31:22.152423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG passlib.handlers.bcrypt [None req-54a4d6d5-a107-489e-80fa-957ecb04f911 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88249) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 07:31:22.206263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-54a4d6d5-a107-489e-80fa-957ecb04f911 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:22.207858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-54a4d6d5-a107-489e-80fa-957ecb04f911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:22.335746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 07:31:22.803970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c893c5-6086-4eef-beb5-45ddc6bc5db7 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:22.872214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: INFO dbcounter [None req-09c893c5-6086-4eef-beb5-45ddc6bc5db7 None None] Registered counter for database keystone Aug 24 07:31:22.891772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG oslo_db.sqlalchemy.engines [None req-09c893c5-6086-4eef-beb5-45ddc6bc5db7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88250) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 07:31:22.898124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writer thread running {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 07:31:23.016865 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c893c5-6086-4eef-beb5-45ddc6bc5db7 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:23.021138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582bbfd-b5fd-432c-a032-ae6e585f4e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:23.021771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582bbfd-b5fd-432c-a032-ae6e585f4e5a admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:23.022571 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582bbfd-b5fd-432c-a032-ae6e585f4e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:23.022846 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582bbfd-b5fd-432c-a032-ae6e585f4e5a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:23.030773 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582bbfd-b5fd-432c-a032-ae6e585f4e5a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:23.082519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f582bbfd-b5fd-432c-a032-ae6e585f4e5a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 07:31:23.082816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582bbfd-b5fd-432c-a032-ae6e585f4e5a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:23.092679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Wed Aug 24 07:31:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 07:31:23.098816 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4d6d5-a107-489e-80fa-957ecb04f911 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:23.113922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4d6d5-a107-489e-80fa-957ecb04f911 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:23.118763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751eff33-9b7c-4dab-81e6-3f107969326f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:23.119854 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eff33-9b7c-4dab-81e6-3f107969326f admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:23.120137 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eff33-9b7c-4dab-81e6-3f107969326f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:23.120408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eff33-9b7c-4dab-81e6-3f107969326f admin admin] PATH_INFO: `/v3/projects/12ef7b65bdcb4d6d980c3b5faea49511` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:23.121988 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751eff33-9b7c-4dab-81e6-3f107969326f admin admin] RBAC: Authorizing `identity:get_project(project_id=12ef7b65bdcb4d6d980c3b5faea49511)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:23.176358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: WARNING keystone.common.rbac_enforcer.enforcer [None req-751eff33-9b7c-4dab-81e6-3f107969326f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 07:31:23.176720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751eff33-9b7c-4dab-81e6-3f107969326f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:23.181503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Wed Aug 24 07:31:23 2022] GET /identity/v3/projects/12ef7b65bdcb4d6d980c3b5faea49511 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:25.064996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2afad6-3743-453b-b43f-0188d3e4f42b None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:25.065514 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2afad6-3743-453b-b43f-0188d3e4f42b None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:25.065600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2afad6-3743-453b-b43f-0188d3e4f42b None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:25.070175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:25.075836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d06288-bcfe-4356-be0d-7029c19f1449 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:25.076227 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d06288-bcfe-4356-be0d-7029c19f1449 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:25.076391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d06288-bcfe-4356-be0d-7029c19f1449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:25.096226 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-77d06288-bcfe-4356-be0d-7029c19f1449 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:25.097769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-77d06288-bcfe-4356-be0d-7029c19f1449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:25.106058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 07:31:25.373293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2afad6-3743-453b-b43f-0188d3e4f42b None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:25.386118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2afad6-3743-453b-b43f-0188d3e4f42b None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:25.390633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b5c0f-e25a-4f34-b644-fd283bd96d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:25.391230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b5c0f-e25a-4f34-b644-fd283bd96d63 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:25.391510 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b5c0f-e25a-4f34-b644-fd283bd96d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:25.391785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b5c0f-e25a-4f34-b644-fd283bd96d63 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:25.399512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b5c0f-e25a-4f34-b644-fd283bd96d63 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:25.401323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b5c0f-e25a-4f34-b644-fd283bd96d63 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:25.409929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Wed Aug 24 07:31:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:31:25.415219 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d06288-bcfe-4356-be0d-7029c19f1449 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:25.424113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d06288-bcfe-4356-be0d-7029c19f1449 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:25.428905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30171bac-229d-459e-8a6e-a9ee232e1e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:25.429526 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30171bac-229d-459e-8a6e-a9ee232e1e34 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:25.429803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30171bac-229d-459e-8a6e-a9ee232e1e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:25.430073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30171bac-229d-459e-8a6e-a9ee232e1e34 admin admin] PATH_INFO: `/v3/users/7275460c8c2e486f8768d1910d8aaa60` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:25.431368 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30171bac-229d-459e-8a6e-a9ee232e1e34 admin admin] RBAC: Authorizing `identity:get_user(user_id=7275460c8c2e486f8768d1910d8aaa60)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:25.433231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30171bac-229d-459e-8a6e-a9ee232e1e34 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:25.434726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Wed Aug 24 07:31:25 2022] GET /identity/v3/users/7275460c8c2e486f8768d1910d8aaa60 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:27.388751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccca31-f107-4973-a66e-7bd653334733 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:27.389331 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccca31-f107-4973-a66e-7bd653334733 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:27.389331 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccca31-f107-4973-a66e-7bd653334733 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:27.390617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:27.395761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141de87b-7d2d-4ecc-9fc4-daa55fef0ad3 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:27.396169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141de87b-7d2d-4ecc-9fc4-daa55fef0ad3 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:27.396456 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141de87b-7d2d-4ecc-9fc4-daa55fef0ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:27.414470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-141de87b-7d2d-4ecc-9fc4-daa55fef0ad3 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:27.415773 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-141de87b-7d2d-4ecc-9fc4-daa55fef0ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:27.435254 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e250e4-c133-4dd2-8ba6-9ba740f24af4 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:27.435543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e250e4-c133-4dd2-8ba6-9ba740f24af4 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:27.435818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e250e4-c133-4dd2-8ba6-9ba740f24af4 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:27.437152 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:27.442186 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d82cd0-86fc-4c22-b1e8-17557916dec4 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:27.442472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d82cd0-86fc-4c22-b1e8-17557916dec4 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:27.442743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d82cd0-86fc-4c22-b1e8-17557916dec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:27.483854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG passlib.handlers.bcrypt [None req-85d82cd0-86fc-4c22-b1e8-17557916dec4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=88250) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 07:31:27.484287 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG passlib.handlers.bcrypt [None req-85d82cd0-86fc-4c22-b1e8-17557916dec4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88250) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 07:31:27.497353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:31:27.530438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-85d82cd0-86fc-4c22-b1e8-17557916dec4 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:27.532038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-85d82cd0-86fc-4c22-b1e8-17557916dec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:27.628634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:31:27.762354 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141de87b-7d2d-4ecc-9fc4-daa55fef0ad3 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:27.786391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:27.865255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141de87b-7d2d-4ecc-9fc4-daa55fef0ad3 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:27.894131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d82cd0-86fc-4c22-b1e8-17557916dec4 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:27.900105 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207394f-9972-46a0-86ea-36fd302fc9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:27.900775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207394f-9972-46a0-86ea-36fd302fc9cc None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:27.901058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207394f-9972-46a0-86ea-36fd302fc9cc None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:27.901335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207394f-9972-46a0-86ea-36fd302fc9cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:27.912303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207394f-9972-46a0-86ea-36fd302fc9cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-d207394f-9972-46a0-86ea-36fd302fc9cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:27.921308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:27.922221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:27.923065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:27.923065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:27.923065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:27.923065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:27.923065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:27.923065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:27.923065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:27.923065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:27.923065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:27.923065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:27.923065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:27.923065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:31:27.923065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:31:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:27.928150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207394f-9972-46a0-86ea-36fd302fc9cc None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:27.998823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d82cd0-86fc-4c22-b1e8-17557916dec4 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:28.001196 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:28.011788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:28.028959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207394f-9972-46a0-86ea-36fd302fc9cc None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:28.035922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ade31-3b99-446e-bf52-1737c460faa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:28.036804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ade31-3b99-446e-bf52-1737c460faa3 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:28.037095 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ade31-3b99-446e-bf52-1737c460faa3 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:28.037372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ade31-3b99-446e-bf52-1737c460faa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:28.063546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6820e-4c20-459b-9cd6-ec0c3c3822d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:28.064149 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6820e-4c20-459b-9cd6-ec0c3c3822d0 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:28.064433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6820e-4c20-459b-9cd6-ec0c3c3822d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:28.064742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6820e-4c20-459b-9cd6-ec0c3c3822d0 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:28.065415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6820e-4c20-459b-9cd6-ec0c3c3822d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:28.065720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6820e-4c20-459b-9cd6-ec0c3c3822d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:28.067161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6820e-4c20-459b-9cd6-ec0c3c3822d0 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:28.079439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:28.085521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6820e-4c20-459b-9cd6-ec0c3c3822d0 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:28.105755 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ade31-3b99-446e-bf52-1737c460faa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:28.107288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ade31-3b99-446e-bf52-1737c460faa3 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:28.181120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6820e-4c20-459b-9cd6-ec0c3c3822d0 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:28.205747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:31:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:31:28.211814 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ade31-3b99-446e-bf52-1737c460faa3 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:28.214591 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:28.215678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc5b54-0af9-4040-887a-a620f19d1689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:28.216318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc5b54-0af9-4040-887a-a620f19d1689 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:28.216599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc5b54-0af9-4040-887a-a620f19d1689 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:28.216905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc5b54-0af9-4040-887a-a620f19d1689 None admin] PATH_INFO: `/v3/users/7275460c8c2e486f8768d1910d8aaa60` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:28.218043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc5b54-0af9-4040-887a-a620f19d1689 None admin] RBAC: Authorizing `identity:get_user(user_id=7275460c8c2e486f8768d1910d8aaa60)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:28.219484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc5b54-0af9-4040-887a-a620f19d1689 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:28.221079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:28 2022] GET /identity/v3/users/7275460c8c2e486f8768d1910d8aaa60 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:28.227427 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc5b54-0af9-4040-887a-a620f19d1689 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:28.242069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:28.314794 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ade31-3b99-446e-bf52-1737c460faa3 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:28.328582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc5b54-0af9-4040-887a-a620f19d1689 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:28.349485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbd1d3-0fd4-46c4-a90d-b113c7cff7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:28.350931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbd1d3-0fd4-46c4-a90d-b113c7cff7ee None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:28.351212 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbd1d3-0fd4-46c4-a90d-b113c7cff7ee None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:28.351490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbd1d3-0fd4-46c4-a90d-b113c7cff7ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:28.363002 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93020590-feae-420c-b50a-6aee4bedbf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:28.363625 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93020590-feae-420c-b50a-6aee4bedbf33 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:28.363900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93020590-feae-420c-b50a-6aee4bedbf33 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:28.364168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93020590-feae-420c-b50a-6aee4bedbf33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:28.365318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93020590-feae-420c-b50a-6aee4bedbf33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:28.367086 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93020590-feae-420c-b50a-6aee4bedbf33 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:28.368869 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 10/18] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:28.369800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbd1d3-0fd4-46c4-a90d-b113c7cff7ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:28.371267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbd1d3-0fd4-46c4-a90d-b113c7cff7ee None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:28.374530 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93020590-feae-420c-b50a-6aee4bedbf33 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-0ccbd1d3-0fd4-46c4-a90d-b113c7cff7ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:31:28.390742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:28.392073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:31:28.392988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:31:28.392988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:31:28.392988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:31:28.392988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 9/19] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:31:28.399740 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbd1d3-0fd4-46c4-a90d-b113c7cff7ee None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:28.455081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:28.455352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:28.474259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93020590-feae-420c-b50a-6aee4bedbf33 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:28.499333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbd1d3-0fd4-46c4-a90d-b113c7cff7ee None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:28.507452 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc63227-6617-4277-b276-d4294a8ef8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:28.508067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc63227-6617-4277-b276-d4294a8ef8f1 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:28.508351 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc63227-6617-4277-b276-d4294a8ef8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:28.508631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc63227-6617-4277-b276-d4294a8ef8f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:28.509389 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc63227-6617-4277-b276-d4294a8ef8f1 None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.domain.id=default, user.id=7275460c8c2e486f8768d1910d8aaa60) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:28.509700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc63227-6617-4277-b276-d4294a8ef8f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:28.511144 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc63227-6617-4277-b276-d4294a8ef8f1 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:28.518169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 11/20] 10.222.0.26 () {56 vars in 1322 bytes} [Wed Aug 24 07:31:28 2022] GET /identity/v3/role_assignments?user.id=7275460c8c2e486f8768d1910d8aaa60&scope.domain.id=default&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:28.534621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36314c99-81fa-4feb-9fa0-9fff14e4413c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:28.535257 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36314c99-81fa-4feb-9fa0-9fff14e4413c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:28.535543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36314c99-81fa-4feb-9fa0-9fff14e4413c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:28.535821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36314c99-81fa-4feb-9fa0-9fff14e4413c None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:28.536491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36314c99-81fa-4feb-9fa0-9fff14e4413c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:28.536841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36314c99-81fa-4feb-9fa0-9fff14e4413c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:28.538885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36314c99-81fa-4feb-9fa0-9fff14e4413c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:28.560263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 10/21] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:30.614865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac3d0c-e784-43ea-88c1-826485e9565d None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:30.615373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac3d0c-e784-43ea-88c1-826485e9565d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:30.615447 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac3d0c-e784-43ea-88c1-826485e9565d None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:30.617215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 12/22] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:30.622579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48a718-548d-4300-a4f3-9b552295fe6b None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:30.622998 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48a718-548d-4300-a4f3-9b552295fe6b None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:30.623185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48a718-548d-4300-a4f3-9b552295fe6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:30.652127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-fc48a718-548d-4300-a4f3-9b552295fe6b None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:30.653354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fc48a718-548d-4300-a4f3-9b552295fe6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:30.750161 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 11/23] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:30.778950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44884d1f-44f2-4b5f-afc9-0734436b8330 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:30.778950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44884d1f-44f2-4b5f-afc9-0734436b8330 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:30.778950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44884d1f-44f2-4b5f-afc9-0734436b8330 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:30.779677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 13/24] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:30.785368 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf0ecf-6a21-416d-8630-a225dadf4ce0 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:30.785888 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf0ecf-6a21-416d-8630-a225dadf4ce0 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:30.786393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf0ecf-6a21-416d-8630-a225dadf4ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:30.810040 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e3bf0ecf-6a21-416d-8630-a225dadf4ce0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:30.811555 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e3bf0ecf-6a21-416d-8630-a225dadf4ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:30.872711 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:30.906993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 12/25] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:31.128739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4203a4a-5200-4695-b2aa-2b6b978833b3 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:31.129127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4203a4a-5200-4695-b2aa-2b6b978833b3 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:31.129315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4203a4a-5200-4695-b2aa-2b6b978833b3 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:31.131036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 14/26] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:31.136490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bea7a0-82f0-46d8-bd15-9e531ff81025 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:31.136813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bea7a0-82f0-46d8-bd15-9e531ff81025 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:31.137100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bea7a0-82f0-46d8-bd15-9e531ff81025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:31.161592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-83bea7a0-82f0-46d8-bd15-9e531ff81025 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:31.163341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-83bea7a0-82f0-46d8-bd15-9e531ff81025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:31.252137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 13/27] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:31.285761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4203a4a-5200-4695-b2aa-2b6b978833b3 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:31.381706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bea7a0-82f0-46d8-bd15-9e531ff81025 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:31.496878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4203a4a-5200-4695-b2aa-2b6b978833b3 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:31.577019 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95709b06-1e97-4022-aec9-586f1a609f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:31.578058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95709b06-1e97-4022-aec9-586f1a609f48 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:31.578534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95709b06-1e97-4022-aec9-586f1a609f48 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:31.578997 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95709b06-1e97-4022-aec9-586f1a609f48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:31.602395 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95709b06-1e97-4022-aec9-586f1a609f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-95709b06-1e97-4022-aec9-586f1a609f48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:31.609558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:31.610421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:31.611170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:31.611170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:31.611170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:31.611170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:31.611170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:31.611170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:31.611170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:31.611170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:31.611170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:31.611170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:31.611170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:31.611170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:31:31.611170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 15/28] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:31:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:31.616940 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0a791-f4b3-4314-9fff-405b2bd00c35 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:31.618553 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0a791-f4b3-4314-9fff-405b2bd00c35 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:31.619028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0a791-f4b3-4314-9fff-405b2bd00c35 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:31.623305 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 16/29] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:31 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:31.628347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0a791-f4b3-4314-9fff-405b2bd00c35 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:31.633366 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bea7a0-82f0-46d8-bd15-9e531ff81025 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:31.638893 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:31.657308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:31.718501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f7b4b-030e-4fe4-83d9-e9d6fa5958d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:31.719438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f7b4b-030e-4fe4-83d9-e9d6fa5958d8 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:31.719905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f7b4b-030e-4fe4-83d9-e9d6fa5958d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:31.720346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f7b4b-030e-4fe4-83d9-e9d6fa5958d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:31.744049 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f7b4b-030e-4fe4-83d9-e9d6fa5958d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:31.745785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f7b4b-030e-4fe4-83d9-e9d6fa5958d8 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-371f7b4b-030e-4fe4-83d9-e9d6fa5958d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:31:31.761796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:31.762905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:31:31.763723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:31:31.763723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:31:31.763723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:31:31.765757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 14/30] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:31.767380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f443f3-936d-4c0a-b476-dddfe522b214 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:31.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f443f3-936d-4c0a-b476-dddfe522b214 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:31.768321 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f443f3-936d-4c0a-b476-dddfe522b214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:31.788097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-70f443f3-936d-4c0a-b476-dddfe522b214 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:31.789733 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-70f443f3-936d-4c0a-b476-dddfe522b214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:31.802125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0a791-f4b3-4314-9fff-405b2bd00c35 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:31.840688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4fd58-22da-4001-b42f-5cf48fe3f32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:31.841423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4fd58-22da-4001-b42f-5cf48fe3f32d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:31.841761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4fd58-22da-4001-b42f-5cf48fe3f32d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:31.842197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4fd58-22da-4001-b42f-5cf48fe3f32d None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:31.842954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4fd58-22da-4001-b42f-5cf48fe3f32d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:31.843680 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4fd58-22da-4001-b42f-5cf48fe3f32d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:31.844960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4fd58-22da-4001-b42f-5cf48fe3f32d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:31.856391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 17/31] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:31.857393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4fd58-22da-4001-b42f-5cf48fe3f32d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:31.870852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 15/32] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:31.872027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f443f3-936d-4c0a-b476-dddfe522b214 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:31.981261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:32.047845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4fd58-22da-4001-b42f-5cf48fe3f32d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:32.049200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:32.078237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f443f3-936d-4c0a-b476-dddfe522b214 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:32.097368 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b625a7c-c29c-4e25-b15f-519b765f853f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:32.098060 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b625a7c-c29c-4e25-b15f-519b765f853f None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:32.098594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b625a7c-c29c-4e25-b15f-519b765f853f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:32.098879 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b625a7c-c29c-4e25-b15f-519b765f853f None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:32.099565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b625a7c-c29c-4e25-b15f-519b765f853f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:32.104666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b625a7c-c29c-4e25-b15f-519b765f853f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:32.133412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99168527-9d09-4c44-a1bd-8c5349460f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:32.134102 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99168527-9d09-4c44-a1bd-8c5349460f59 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:32.134382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99168527-9d09-4c44-a1bd-8c5349460f59 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:32.134716 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99168527-9d09-4c44-a1bd-8c5349460f59 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:32.135372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99168527-9d09-4c44-a1bd-8c5349460f59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:32.135677 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99168527-9d09-4c44-a1bd-8c5349460f59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:32.137445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99168527-9d09-4c44-a1bd-8c5349460f59 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:32.152654 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 16/33] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:32.153741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99168527-9d09-4c44-a1bd-8c5349460f59 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:32.279551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99168527-9d09-4c44-a1bd-8c5349460f59 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:32.318252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e23061-90fc-4e19-9fdb-385867e93b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:32.319046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e23061-90fc-4e19-9fdb-385867e93b4c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:32.319508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e23061-90fc-4e19-9fdb-385867e93b4c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:32.319800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e23061-90fc-4e19-9fdb-385867e93b4c None admin] PATH_INFO: `/v3/users/7275460c8c2e486f8768d1910d8aaa60` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:32.321326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e23061-90fc-4e19-9fdb-385867e93b4c None admin] RBAC: Authorizing `identity:get_user(user_id=7275460c8c2e486f8768d1910d8aaa60)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:32.322772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e23061-90fc-4e19-9fdb-385867e93b4c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:32.324361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 17/34] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:32 2022] GET /identity/v3/users/7275460c8c2e486f8768d1910d8aaa60 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:32.328968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e23061-90fc-4e19-9fdb-385867e93b4c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:32.329048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 18/35] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Aug 24 07:31:31 2022] POST /identity/v3/roles => generated 219 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:31:32.329608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b625a7c-c29c-4e25-b15f-519b765f853f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:32.357899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:32.443105 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b625a7c-c29c-4e25-b15f-519b765f853f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:32.466672 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e23061-90fc-4e19-9fdb-385867e93b4c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:32.472187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:32.481927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19984d90-efe1-403c-af12-3507236da5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:32.482551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19984d90-efe1-403c-af12-3507236da5d0 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:32.485487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19984d90-efe1-403c-af12-3507236da5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:32.485487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19984d90-efe1-403c-af12-3507236da5d0 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:32.485487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19984d90-efe1-403c-af12-3507236da5d0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:32.486101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19984d90-efe1-403c-af12-3507236da5d0 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:32.510742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db455f-cd74-4bcb-a5f9-c226284d2398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:32.511355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db455f-cd74-4bcb-a5f9-c226284d2398 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:32.514770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db455f-cd74-4bcb-a5f9-c226284d2398 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:32.514770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db455f-cd74-4bcb-a5f9-c226284d2398 None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:32.514770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db455f-cd74-4bcb-a5f9-c226284d2398 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:32.514770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db455f-cd74-4bcb-a5f9-c226284d2398 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:32.590882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 19/36] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Aug 24 07:31:32 2022] POST /identity/v3/roles => generated 225 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:31:32.592343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f475d9e-6937-4ec5-95b2-52e62996cb4c None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:32.592662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f475d9e-6937-4ec5-95b2-52e62996cb4c None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:32.592947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f475d9e-6937-4ec5-95b2-52e62996cb4c None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:32.594171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 20/37] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:32.595529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f475d9e-6937-4ec5-95b2-52e62996cb4c None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:32.714587 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f475d9e-6937-4ec5-95b2-52e62996cb4c None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:32.751409 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 18/38] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 24 07:31:32 2022] POST /identity/v3/projects => generated 301 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:31:32.752896 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c7f55-3a2f-491d-ace6-72cb8f53dfd9 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:32.753187 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c7f55-3a2f-491d-ace6-72cb8f53dfd9 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:32.753466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c7f55-3a2f-491d-ace6-72cb8f53dfd9 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:32.754719 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 19/39] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:32.755774 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb51783-5f05-49fa-8797-1e3ad1e35824 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:32.756179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb51783-5f05-49fa-8797-1e3ad1e35824 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:32.756636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb51783-5f05-49fa-8797-1e3ad1e35824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:32.760603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea02a2-0c09-4d0c-83c3-2a2c6154847f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:32.762287 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea02a2-0c09-4d0c-83c3-2a2c6154847f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:32.762762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea02a2-0c09-4d0c-83c3-2a2c6154847f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:32.763217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea02a2-0c09-4d0c-83c3-2a2c6154847f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:32.764799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea02a2-0c09-4d0c-83c3-2a2c6154847f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:32.766286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea02a2-0c09-4d0c-83c3-2a2c6154847f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:32.768334 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 21/40] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:32.769603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b25d77-9862-4e3f-bf44-b08e4c4a600f None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:32.769991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b25d77-9862-4e3f-bf44-b08e4c4a600f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:32.770277 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b25d77-9862-4e3f-bf44-b08e4c4a600f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:32.782768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-3bb51783-5f05-49fa-8797-1e3ad1e35824 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:32.784365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-3bb51783-5f05-49fa-8797-1e3ad1e35824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:32.795063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-76b25d77-9862-4e3f-bf44-b08e4c4a600f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:32.797010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-76b25d77-9862-4e3f-bf44-b08e4c4a600f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:32.862696 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:32.873457 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:32.879334 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 20/41] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:32.884080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb51783-5f05-49fa-8797-1e3ad1e35824 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:32.886350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 22/42] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:32.992435 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb51783-5f05-49fa-8797-1e3ad1e35824 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:33.029158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ac248-e4ca-4010-8295-f46815fb3477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:33.029827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ac248-e4ca-4010-8295-f46815fb3477 None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:33.030115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ac248-e4ca-4010-8295-f46815fb3477 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:33.030398 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ac248-e4ca-4010-8295-f46815fb3477 None admin] PATH_INFO: `/v3/domains/default/users/7275460c8c2e486f8768d1910d8aaa60/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:33.032331 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ac248-e4ca-4010-8295-f46815fb3477 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7275460c8c2e486f8768d1910d8aaa60, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:33.034001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ac248-e4ca-4010-8295-f46815fb3477 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:33.144329 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 21/43] 10.222.0.26 () {58 vars in 1361 bytes} [Wed Aug 24 07:31:32 2022] PUT /identity/v3/domains/default/users/7275460c8c2e486f8768d1910d8aaa60/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:31:33.160205 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b25d77-9862-4e3f-bf44-b08e4c4a600f None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:33.170248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ac248-e4ca-4010-8295-f46815fb3477 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:33.234344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:33.275804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b25d77-9862-4e3f-bf44-b08e4c4a600f None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:33.282430 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ac248-e4ca-4010-8295-f46815fb3477 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:33.315826 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c237160-415f-4cc0-a14f-d823f8d2c7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:33.316444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c237160-415f-4cc0-a14f-d823f8d2c7e2 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:33.316830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c237160-415f-4cc0-a14f-d823f8d2c7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:33.317111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c237160-415f-4cc0-a14f-d823f8d2c7e2 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:33.317821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c237160-415f-4cc0-a14f-d823f8d2c7e2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:33.319541 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c237160-415f-4cc0-a14f-d823f8d2c7e2 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:33.322226 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375176fa-0a05-4467-aba7-19e024709d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:33.323036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375176fa-0a05-4467-aba7-19e024709d46 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:33.323322 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375176fa-0a05-4467-aba7-19e024709d46 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:33.323996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375176fa-0a05-4467-aba7-19e024709d46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:33.325146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375176fa-0a05-4467-aba7-19e024709d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:33.326584 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375176fa-0a05-4467-aba7-19e024709d46 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:33.328138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 22/44] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:33.333643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375176fa-0a05-4467-aba7-19e024709d46 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:33.390192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 23/45] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Aug 24 07:31:33 2022] POST /identity/v3/roles => generated 223 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:31:33.439352 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375176fa-0a05-4467-aba7-19e024709d46 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:33.468440 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:33.478501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5261a1-b39e-49d6-bc65-8f89d26de16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:33.479124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5261a1-b39e-49d6-bc65-8f89d26de16b None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:33.479402 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5261a1-b39e-49d6-bc65-8f89d26de16b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:33.479675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5261a1-b39e-49d6-bc65-8f89d26de16b None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:33.480358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5261a1-b39e-49d6-bc65-8f89d26de16b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:33.481854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5261a1-b39e-49d6-bc65-8f89d26de16b None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:33.569831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 23/46] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 24 07:31:33 2022] POST /identity/v3/projects => generated 312 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:31:35.045458 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d348f16-9af7-4eed-9a5e-3d7e1753e1d3 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:35.045981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d348f16-9af7-4eed-9a5e-3d7e1753e1d3 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:35.046064 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d348f16-9af7-4eed-9a5e-3d7e1753e1d3 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:35.047352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 24/47] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:35.052843 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076bab6-7d08-4ad9-afc2-ad3bfed7d395 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:35.053951 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076bab6-7d08-4ad9-afc2-ad3bfed7d395 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:35.054327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076bab6-7d08-4ad9-afc2-ad3bfed7d395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:35.072792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-5076bab6-7d08-4ad9-afc2-ad3bfed7d395 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:35.073994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5076bab6-7d08-4ad9-afc2-ad3bfed7d395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:35.139706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 24/48] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:35.408254 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d348f16-9af7-4eed-9a5e-3d7e1753e1d3 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:35.455630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:35.475539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1ea6e-ad6f-4e28-9f91-7cc9af9c6a02 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:35.475891 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1ea6e-ad6f-4e28-9f91-7cc9af9c6a02 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:35.476286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1ea6e-ad6f-4e28-9f91-7cc9af9c6a02 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:35.477684 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 25/49] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:35.482496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ed9fe-bcd4-4772-8e73-ce6eeb555a74 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:35.482784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ed9fe-bcd4-4772-8e73-ce6eeb555a74 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:35.483067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ed9fe-bcd4-4772-8e73-ce6eeb555a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:35.502747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d43ed9fe-bcd4-4772-8e73-ce6eeb555a74 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:35.504034 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d43ed9fe-bcd4-4772-8e73-ce6eeb555a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:35.578204 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 26/50] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:31:35.617284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d348f16-9af7-4eed-9a5e-3d7e1753e1d3 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:35.650687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43124db0-a749-4fe1-94a1-336fecc04b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:35.651312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43124db0-a749-4fe1-94a1-336fecc04b31 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:35.651596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43124db0-a749-4fe1-94a1-336fecc04b31 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:35.651873 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43124db0-a749-4fe1-94a1-336fecc04b31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:35.661709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43124db0-a749-4fe1-94a1-336fecc04b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-43124db0-a749-4fe1-94a1-336fecc04b31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:35.666881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:35.667717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:35.668454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:35.668454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:35.668454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:35.668454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:35.668454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:35.668454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:35.668454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:35.668454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:35.668454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:35.668454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:35.668454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:35.668454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:31:35.668454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 25/51] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:31:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:35.673885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ed9fe-bcd4-4772-8e73-ce6eeb555a74 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:35.713074 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:35.771468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ed9fe-bcd4-4772-8e73-ce6eeb555a74 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:35.805541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20e418-1491-4f26-acc7-40f595fe35fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:35.806146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e418-1491-4f26-acc7-40f595fe35fc None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:35.806431 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e418-1491-4f26-acc7-40f595fe35fc None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:35.806708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e418-1491-4f26-acc7-40f595fe35fc None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:35.807377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20e418-1491-4f26-acc7-40f595fe35fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:35.807688 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20e418-1491-4f26-acc7-40f595fe35fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:35.809106 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20e418-1491-4f26-acc7-40f595fe35fc None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:35.820837 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 27/52] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:35.826256 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43124db0-a749-4fe1-94a1-336fecc04b31 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:35.847649 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20e418-1491-4f26-acc7-40f595fe35fc None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:35.925859 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43124db0-a749-4fe1-94a1-336fecc04b31 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:35.928046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:35.944862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:35.947694 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20e418-1491-4f26-acc7-40f595fe35fc None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:35.958881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08269033-5c06-49f9-ab9d-7323cd7f9de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:35.960148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08269033-5c06-49f9-ab9d-7323cd7f9de6 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:35.960521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08269033-5c06-49f9-ab9d-7323cd7f9de6 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:35.960830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08269033-5c06-49f9-ab9d-7323cd7f9de6 None admin] PATH_INFO: `/v3/users/7275460c8c2e486f8768d1910d8aaa60` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:35.961966 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08269033-5c06-49f9-ab9d-7323cd7f9de6 None admin] RBAC: Authorizing `identity:get_user(user_id=7275460c8c2e486f8768d1910d8aaa60)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:35.963408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08269033-5c06-49f9-ab9d-7323cd7f9de6 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:35.965002 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 26/53] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:35 2022] GET /identity/v3/users/7275460c8c2e486f8768d1910d8aaa60 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:35.970591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08269033-5c06-49f9-ab9d-7323cd7f9de6 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:35.982316 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59ff3b-87a8-421b-b6d7-20370493ba66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:35.982954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59ff3b-87a8-421b-b6d7-20370493ba66 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:35.983243 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59ff3b-87a8-421b-b6d7-20370493ba66 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:35.983526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59ff3b-87a8-421b-b6d7-20370493ba66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:35.984671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59ff3b-87a8-421b-b6d7-20370493ba66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:35.986148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59ff3b-87a8-421b-b6d7-20370493ba66 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:35.987958 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 28/54] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:35.993857 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59ff3b-87a8-421b-b6d7-20370493ba66 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:36.071668 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08269033-5c06-49f9-ab9d-7323cd7f9de6 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:36.092831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59ff3b-87a8-421b-b6d7-20370493ba66 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:36.106236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f618c22-fe5b-4bca-9b23-113dcca6f936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:36.106868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f618c22-fe5b-4bca-9b23-113dcca6f936 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:36.107155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f618c22-fe5b-4bca-9b23-113dcca6f936 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:36.107432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f618c22-fe5b-4bca-9b23-113dcca6f936 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:36.108558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f618c22-fe5b-4bca-9b23-113dcca6f936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:36.117217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f618c22-fe5b-4bca-9b23-113dcca6f936 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:36.117217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 27/55] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:36.117501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f618c22-fe5b-4bca-9b23-113dcca6f936 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:36.127208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146552f-2f92-420c-b12c-859e493f9ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:36.127825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146552f-2f92-420c-b12c-859e493f9ca9 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:36.128115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146552f-2f92-420c-b12c-859e493f9ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:36.128396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146552f-2f92-420c-b12c-859e493f9ca9 None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:36.129071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146552f-2f92-420c-b12c-859e493f9ca9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:36.130526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146552f-2f92-420c-b12c-859e493f9ca9 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:36.158420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:36.213788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f618c22-fe5b-4bca-9b23-113dcca6f936 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:36.246878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00402f70-ac4f-4e1b-b2c2-c496acfff983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:36.247475 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00402f70-ac4f-4e1b-b2c2-c496acfff983 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:36.247761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00402f70-ac4f-4e1b-b2c2-c496acfff983 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:36.248040 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00402f70-ac4f-4e1b-b2c2-c496acfff983 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:36.248790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00402f70-ac4f-4e1b-b2c2-c496acfff983 None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.domain.id=default, user.id=7275460c8c2e486f8768d1910d8aaa60) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:36.249101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00402f70-ac4f-4e1b-b2c2-c496acfff983 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:36.250537 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00402f70-ac4f-4e1b-b2c2-c496acfff983 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:36.256700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 28/56] 10.222.0.26 () {56 vars in 1322 bytes} [Wed Aug 24 07:31:36 2022] GET /identity/v3/role_assignments?user.id=7275460c8c2e486f8768d1910d8aaa60&scope.domain.id=default&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 530 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:36.289219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 29/57] 10.222.0.26 () {60 vars in 1140 bytes} [Wed Aug 24 07:31:35 2022] POST /identity/v3/projects => generated 298 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:31:38.194774 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e756b2-38e6-4961-a79e-901398fba927 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:38.195302 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e756b2-38e6-4961-a79e-901398fba927 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:38.195403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e756b2-38e6-4961-a79e-901398fba927 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:38.196700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 29/58] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:38.201827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1ecd5-51c2-489c-adfc-b058da1c2a5d None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:38.202259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1ecd5-51c2-489c-adfc-b058da1c2a5d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:38.202380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1ecd5-51c2-489c-adfc-b058da1c2a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:38.220552 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-5ae1ecd5-51c2-489c-adfc-b058da1c2a5d None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:38.221769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5ae1ecd5-51c2-489c-adfc-b058da1c2a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:38.235562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:38.286777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 30/59] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:38.554813 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e756b2-38e6-4961-a79e-901398fba927 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:38.655325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e756b2-38e6-4961-a79e-901398fba927 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:38.657545 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:38.688138 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf7dcc-ca93-4e84-9576-f437270c35f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:38.688783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf7dcc-ca93-4e84-9576-f437270c35f5 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:38.689068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf7dcc-ca93-4e84-9576-f437270c35f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:38.689344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf7dcc-ca93-4e84-9576-f437270c35f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:38.690445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf7dcc-ca93-4e84-9576-f437270c35f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:38.691905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf7dcc-ca93-4e84-9576-f437270c35f5 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:38.693899 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 30/60] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:38.699232 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1ecd5-51c2-489c-adfc-b058da1c2a5d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:38.796432 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1ecd5-51c2-489c-adfc-b058da1c2a5d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:38.828142 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cea86-ce36-4ad5-97c2-fc7775a50c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:38.828765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cea86-ce36-4ad5-97c2-fc7775a50c87 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:38.829049 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cea86-ce36-4ad5-97c2-fc7775a50c87 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:38.829324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cea86-ce36-4ad5-97c2-fc7775a50c87 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:38.829959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cea86-ce36-4ad5-97c2-fc7775a50c87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:38.831405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cea86-ce36-4ad5-97c2-fc7775a50c87 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:38.944471 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 31/61] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:31:38 2022] POST /identity/v3/users => generated 275 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:31:41.837318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67cf68-3444-4e32-8250-6d9187dd0d99 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:41.837813 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67cf68-3444-4e32-8250-6d9187dd0d99 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:41.837914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67cf68-3444-4e32-8250-6d9187dd0d99 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:41.839521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 31/62] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:41.844370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0783d7d-c825-47c0-a6fa-ef7ced87bca4 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:41.844782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0783d7d-c825-47c0-a6fa-ef7ced87bca4 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:41.844959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0783d7d-c825-47c0-a6fa-ef7ced87bca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:41.862064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:41.990091 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-b0783d7d-c825-47c0-a6fa-ef7ced87bca4 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:41.992066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b0783d7d-c825-47c0-a6fa-ef7ced87bca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:42.041910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c05c1-51e0-4880-b5e6-84340de7db85 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:42.042221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c05c1-51e0-4880-b5e6-84340de7db85 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:42.042471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c05c1-51e0-4880-b5e6-84340de7db85 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:42.044093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 32/63] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:42.049263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00629b-b45d-4648-a80c-e512bb844624 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:42.049557 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00629b-b45d-4648-a80c-e512bb844624 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:42.049839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00629b-b45d-4648-a80c-e512bb844624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:42.076827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-ab00629b-b45d-4648-a80c-e512bb844624 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:42.078344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ab00629b-b45d-4648-a80c-e512bb844624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:42.120785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 32/64] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:31:42.123123 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a72b8-024f-4763-bcf9-78c0baa3bcbe None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:42.123833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a72b8-024f-4763-bcf9-78c0baa3bcbe None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:42.124730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a72b8-024f-4763-bcf9-78c0baa3bcbe None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:42.126625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 33/65] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:42.132289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9d71-6b44-472e-bdcf-eba27dfc64cc None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:42.132864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9d71-6b44-472e-bdcf-eba27dfc64cc None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:42.133344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9d71-6b44-472e-bdcf-eba27dfc64cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:42.151388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-aa6e9d71-6b44-472e-bdcf-eba27dfc64cc None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:42.152961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-aa6e9d71-6b44-472e-bdcf-eba27dfc64cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:42.167151 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 33/66] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:31:42.226337 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 34/67] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:31:42.465389 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00629b-b45d-4648-a80c-e512bb844624 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:42.564818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9d71-6b44-472e-bdcf-eba27dfc64cc None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:42.587436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00629b-b45d-4648-a80c-e512bb844624 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:42.589054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:42.623556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deed74c-245c-433f-ba61-b0139399eaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:42.624198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deed74c-245c-433f-ba61-b0139399eaeb None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:42.624485 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deed74c-245c-433f-ba61-b0139399eaeb None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:42.624796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deed74c-245c-433f-ba61-b0139399eaeb None admin] PATH_INFO: `/v3/roles/member` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:42.635492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deed74c-245c-433f-ba61-b0139399eaeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:42.637479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-0deed74c-245c-433f-ba61-b0139399eaeb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:42.640250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:42.641343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:42.642094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:42.642094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:42.642094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:42.642094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:42.642094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:42.642094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:42.642094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:42.642094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:42.642094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:42.642094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:42.642094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:31:42.642094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:31:42.642094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 34/68] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:42.642716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deed74c-245c-433f-ba61-b0139399eaeb None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:42.678616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9d71-6b44-472e-bdcf-eba27dfc64cc None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:42.719584 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de5208-364b-4ada-aef6-1f6fb4d2edae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:42.720137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de5208-364b-4ada-aef6-1f6fb4d2edae None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:42.720423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de5208-364b-4ada-aef6-1f6fb4d2edae None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:42.720729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de5208-364b-4ada-aef6-1f6fb4d2edae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:42.734063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de5208-364b-4ada-aef6-1f6fb4d2edae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-e6de5208-364b-4ada-aef6-1f6fb4d2edae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:42.739937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:42.741579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:42.742487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:42.742487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:42.742487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:42.742487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:42.742487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:42.742487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:42.742487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:42.742487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:42.742487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:42.742487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:42.742487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:31:42.742487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:31:42.742487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 35/69] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:31:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:42.742487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de5208-364b-4ada-aef6-1f6fb4d2edae None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:42.750286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deed74c-245c-433f-ba61-b0139399eaeb None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:42.784576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710617b7-8669-4ecd-8c77-7e717d8ebcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:42.785247 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710617b7-8669-4ecd-8c77-7e717d8ebcd0 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:42.785534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710617b7-8669-4ecd-8c77-7e717d8ebcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:42.785809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710617b7-8669-4ecd-8c77-7e717d8ebcd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:42.796347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710617b7-8669-4ecd-8c77-7e717d8ebcd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-710617b7-8669-4ecd-8c77-7e717d8ebcd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:42.801121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:42.801956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:42.802727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:42.802727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:42.802727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:42.802727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:42.802727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:42.802727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:42.802727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:42.802727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:42.802727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:42.802727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:42.802727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:42.802727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:31:42.802727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 35/70] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:31:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:42.804362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea7957-6f46-448f-a77a-cee3072e6dfe None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:42.804362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea7957-6f46-448f-a77a-cee3072e6dfe None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:42.804659 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea7957-6f46-448f-a77a-cee3072e6dfe None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:42.806038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 36/71] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:42.806906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d9151-632a-47da-8067-67191665ddb6 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:42.807190 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d9151-632a-47da-8067-67191665ddb6 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:42.807468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d9151-632a-47da-8067-67191665ddb6 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:42.809182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 37/72] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:42.809890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d9151-632a-47da-8067-67191665ddb6 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:42.834927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:42.850958 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de5208-364b-4ada-aef6-1f6fb4d2edae None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:42.876465 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:42.884599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a6d66-c521-4fdc-855c-d19712d295ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:42.885373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a6d66-c521-4fdc-855c-d19712d295ad None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:42.885656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a6d66-c521-4fdc-855c-d19712d295ad None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:42.885934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a6d66-c521-4fdc-855c-d19712d295ad None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:42.886598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a6d66-c521-4fdc-855c-d19712d295ad None admin] RBAC: Adding query filter params (name=member) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:42.886907 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a6d66-c521-4fdc-855c-d19712d295ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:42.888303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a6d66-c521-4fdc-855c-d19712d295ad None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:42.896299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 36/73] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:31:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:42.897144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a6d66-c521-4fdc-855c-d19712d295ad None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:42.914085 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d9151-632a-47da-8067-67191665ddb6 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:42.947871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d5cee-4e97-4b42-a469-55a05726f71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:42.948479 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d5cee-4e97-4b42-a469-55a05726f71f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:42.948783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d5cee-4e97-4b42-a469-55a05726f71f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:42.949057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d5cee-4e97-4b42-a469-55a05726f71f None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:42.949721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d5cee-4e97-4b42-a469-55a05726f71f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:42.950024 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d5cee-4e97-4b42-a469-55a05726f71f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:42.951394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d5cee-4e97-4b42-a469-55a05726f71f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:42.959308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 38/74] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:31:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:42.960798 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142fcd1-6c50-4f11-ba25-ff35120c0eb4 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:42.960798 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142fcd1-6c50-4f11-ba25-ff35120c0eb4 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:42.961080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142fcd1-6c50-4f11-ba25-ff35120c0eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:42.978914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-d142fcd1-6c50-4f11-ba25-ff35120c0eb4 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:42.980063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d142fcd1-6c50-4f11-ba25-ff35120c0eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:43.002205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a6d66-c521-4fdc-855c-d19712d295ad None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:43.035724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaac7a8-d33a-4b70-ac53-fbc96db45fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:43.036334 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaac7a8-d33a-4b70-ac53-fbc96db45fea None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:43.036617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaac7a8-d33a-4b70-ac53-fbc96db45fea None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:43.036930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaac7a8-d33a-4b70-ac53-fbc96db45fea None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:43.038209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaac7a8-d33a-4b70-ac53-fbc96db45fea None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:43.038518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaac7a8-d33a-4b70-ac53-fbc96db45fea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:43.039913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaac7a8-d33a-4b70-ac53-fbc96db45fea None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:43.047913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 37/75] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:43.049113 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37d549-c987-41a4-85e0-80b06d378161 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:43.049592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37d549-c987-41a4-85e0-80b06d378161 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:43.049693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 39/76] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:43.049767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37d549-c987-41a4-85e0-80b06d378161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:43.050382 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142fcd1-6c50-4f11-ba25-ff35120c0eb4 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:43.067844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-6a37d549-c987-41a4-85e0-80b06d378161 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:43.069490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6a37d549-c987-41a4-85e0-80b06d378161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:43.073454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:43.119759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:43.144044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 38/77] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:43.145169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37d549-c987-41a4-85e0-80b06d378161 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:43.164985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142fcd1-6c50-4f11-ba25-ff35120c0eb4 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:43.213762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a2508-5c7e-4f8a-ac1d-4d48bda98ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:43.215260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a2508-5c7e-4f8a-ac1d-4d48bda98ed7 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:43.216144 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a2508-5c7e-4f8a-ac1d-4d48bda98ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:43.216964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a2508-5c7e-4f8a-ac1d-4d48bda98ed7 None admin] PATH_INFO: `/v3/users/54d0a424442447c49a385a116cc9eec2` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:43.236846 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a2508-5c7e-4f8a-ac1d-4d48bda98ed7 None admin] RBAC: Authorizing `identity:get_user(user_id=54d0a424442447c49a385a116cc9eec2)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:43.238810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a2508-5c7e-4f8a-ac1d-4d48bda98ed7 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:43.242126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 40/78] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:43 2022] GET /identity/v3/users/54d0a424442447c49a385a116cc9eec2 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:43.243103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a2508-5c7e-4f8a-ac1d-4d48bda98ed7 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:43.282428 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37d549-c987-41a4-85e0-80b06d378161 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:43.325056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75048ff-b1c6-43e3-923c-f5d710fed795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:43.325959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75048ff-b1c6-43e3-923c-f5d710fed795 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:43.326243 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75048ff-b1c6-43e3-923c-f5d710fed795 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:43.326564 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75048ff-b1c6-43e3-923c-f5d710fed795 None admin] PATH_INFO: `/v3/users/54d0a424442447c49a385a116cc9eec2` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:43.327648 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75048ff-b1c6-43e3-923c-f5d710fed795 None admin] RBAC: Authorizing `identity:get_user(user_id=54d0a424442447c49a385a116cc9eec2)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:43.329134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75048ff-b1c6-43e3-923c-f5d710fed795 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:43.330835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 39/79] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:43 2022] GET /identity/v3/users/54d0a424442447c49a385a116cc9eec2 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:43.333601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75048ff-b1c6-43e3-923c-f5d710fed795 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:43.336981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:43.367072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a2508-5c7e-4f8a-ac1d-4d48bda98ed7 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:43.386279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:43.404197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb84b68-efd5-4da6-a7ee-436875170284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:43.404886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb84b68-efd5-4da6-a7ee-436875170284 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:43.405189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb84b68-efd5-4da6-a7ee-436875170284 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:43.405576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb84b68-efd5-4da6-a7ee-436875170284 None admin] PATH_INFO: `/v3/users/7275460c8c2e486f8768d1910d8aaa60` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:43.406762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb84b68-efd5-4da6-a7ee-436875170284 None admin] RBAC: Authorizing `identity:get_user(user_id=7275460c8c2e486f8768d1910d8aaa60)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:43.408206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb84b68-efd5-4da6-a7ee-436875170284 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:43.414680 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 41/80] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:43 2022] GET /identity/v3/users/7275460c8c2e486f8768d1910d8aaa60 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:43.414680 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb84b68-efd5-4da6-a7ee-436875170284 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:43.441156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75048ff-b1c6-43e3-923c-f5d710fed795 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:43.475728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a6289-bfaf-43ae-b674-2590026c905a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:43.476276 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a6289-bfaf-43ae-b674-2590026c905a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:43.476553 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a6289-bfaf-43ae-b674-2590026c905a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:43.476854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a6289-bfaf-43ae-b674-2590026c905a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:43.513337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb84b68-efd5-4da6-a7ee-436875170284 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:43.543840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a6289-bfaf-43ae-b674-2590026c905a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:43.545356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a6289-bfaf-43ae-b674-2590026c905a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:43.546835 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec6202-45ea-4bee-a84d-156ea8a544e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:43.547470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec6202-45ea-4bee-a84d-156ea8a544e9 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:43.547746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec6202-45ea-4bee-a84d-156ea8a544e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:43.548031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec6202-45ea-4bee-a84d-156ea8a544e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:43.549179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec6202-45ea-4bee-a84d-156ea8a544e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:43.550616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec6202-45ea-4bee-a84d-156ea8a544e9 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:43.552183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 42/81] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:43.553614 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec6202-45ea-4bee-a84d-156ea8a544e9 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:43.564006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:43.599373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:43.651089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec6202-45ea-4bee-a84d-156ea8a544e9 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:43.655770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 40/82] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:31:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:31:43.655770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a6289-bfaf-43ae-b674-2590026c905a None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:43.688888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7c360-09e4-4a7f-8756-5425e2172afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:43.689499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7c360-09e4-4a7f-8756-5425e2172afa None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:43.689787 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7c360-09e4-4a7f-8756-5425e2172afa None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:43.690070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7c360-09e4-4a7f-8756-5425e2172afa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:43.750597 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a6289-bfaf-43ae-b674-2590026c905a None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:43.755545 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7c360-09e4-4a7f-8756-5425e2172afa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:43.757112 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7c360-09e4-4a7f-8756-5425e2172afa None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:43.766388 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:43.784976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed11a64-3cdd-4e96-aafb-743daea9bfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:43.785572 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed11a64-3cdd-4e96-aafb-743daea9bfc9 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:43.785855 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed11a64-3cdd-4e96-aafb-743daea9bfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:43.786130 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed11a64-3cdd-4e96-aafb-743daea9bfc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:43.805031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:43.854440 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed11a64-3cdd-4e96-aafb-743daea9bfc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:43.855922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed11a64-3cdd-4e96-aafb-743daea9bfc9 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:43.861374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 43/83] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:31:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:31:43.861374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7c360-09e4-4a7f-8756-5425e2172afa None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:43.957938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7c360-09e4-4a7f-8756-5425e2172afa None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:43.960104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 41/84] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:31:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:31:43.960961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed11a64-3cdd-4e96-aafb-743daea9bfc9 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:43.972507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:43.998405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ef5b0-c3c6-4305-95b6-ec8cd1dec9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:43.999033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ef5b0-c3c6-4305-95b6-ec8cd1dec9db None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:43.999317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ef5b0-c3c6-4305-95b6-ec8cd1dec9db None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:43.999597 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ef5b0-c3c6-4305-95b6-ec8cd1dec9db None admin] PATH_INFO: `/v3/roles/member` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:44.008962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:44.010689 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ef5b0-c3c6-4305-95b6-ec8cd1dec9db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-197ef5b0-c3c6-4305-95b6-ec8cd1dec9db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:44.015370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:44.018154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:44.018944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:44.018944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:44.018944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:44.018944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:44.018944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:44.018944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:44.018944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:44.018944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:44.018944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:44.018944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:44.018944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:31:44.018944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:31:44.018944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 44/85] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:44.018944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ef5b0-c3c6-4305-95b6-ec8cd1dec9db None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:44.060482 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed11a64-3cdd-4e96-aafb-743daea9bfc9 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:44.094949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bb314-4766-40a8-be3c-94934ec46ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:44.095566 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bb314-4766-40a8-be3c-94934ec46ea1 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:44.095848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bb314-4766-40a8-be3c-94934ec46ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:44.096126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bb314-4766-40a8-be3c-94934ec46ea1 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:44.096808 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360bb314-4766-40a8-be3c-94934ec46ea1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:44.098503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360bb314-4766-40a8-be3c-94934ec46ea1 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:44.120183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ef5b0-c3c6-4305-95b6-ec8cd1dec9db None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:44.153575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c309e99-5081-4819-8c12-4d816a178149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:44.154207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c309e99-5081-4819-8c12-4d816a178149 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:44.154612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c309e99-5081-4819-8c12-4d816a178149 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:44.154874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c309e99-5081-4819-8c12-4d816a178149 None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:44.156015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c309e99-5081-4819-8c12-4d816a178149 None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:44.157574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c309e99-5081-4819-8c12-4d816a178149 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:44.159201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 45/86] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:44 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:44.160050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c309e99-5081-4819-8c12-4d816a178149 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:44.177854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 42/87] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:31:43 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:31:44.177854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bb314-4766-40a8-be3c-94934ec46ea1 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:44.214283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:44.287517 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c309e99-5081-4819-8c12-4d816a178149 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:44.287765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bb314-4766-40a8-be3c-94934ec46ea1 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:44.291818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:44.327976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494d898-cb01-48d7-bf06-97ce8ae3dc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:44.328626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494d898-cb01-48d7-bf06-97ce8ae3dc52 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:44.328956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494d898-cb01-48d7-bf06-97ce8ae3dc52 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:44.329236 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494d898-cb01-48d7-bf06-97ce8ae3dc52 None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:44.330487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494d898-cb01-48d7-bf06-97ce8ae3dc52 None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:44.331933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494d898-cb01-48d7-bf06-97ce8ae3dc52 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:44.333309 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976e0a2-e5c0-4bfd-bf37-1647f048421b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:44.334018 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 43/88] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:44 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:44.334149 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976e0a2-e5c0-4bfd-bf37-1647f048421b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:44.334553 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976e0a2-e5c0-4bfd-bf37-1647f048421b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:44.334871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976e0a2-e5c0-4bfd-bf37-1647f048421b None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:44.334958 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494d898-cb01-48d7-bf06-97ce8ae3dc52 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:44.336079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976e0a2-e5c0-4bfd-bf37-1647f048421b None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:44.337551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976e0a2-e5c0-4bfd-bf37-1647f048421b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:44.339413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 46/89] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:44 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:44.340117 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976e0a2-e5c0-4bfd-bf37-1647f048421b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:44.460531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494d898-cb01-48d7-bf06-97ce8ae3dc52 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:44.460740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976e0a2-e5c0-4bfd-bf37-1647f048421b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:44.465971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:44.505376 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df08903-6d27-48d9-b0c4-51fc787899dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:44.506531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df08903-6d27-48d9-b0c4-51fc787899dc None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:44.506817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df08903-6d27-48d9-b0c4-51fc787899dc None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:44.507139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df08903-6d27-48d9-b0c4-51fc787899dc None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:44.507790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df08903-6d27-48d9-b0c4-51fc787899dc None admin] RBAC: Adding query filter params (name=member) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:44.508099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df08903-6d27-48d9-b0c4-51fc787899dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:44.508249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166b694-da08-4689-a71e-552802e97830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:44.509310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b694-da08-4689-a71e-552802e97830 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:44.509310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b694-da08-4689-a71e-552802e97830 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:44.509461 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b694-da08-4689-a71e-552802e97830 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:44.509544 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df08903-6d27-48d9-b0c4-51fc787899dc None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:44.510159 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166b694-da08-4689-a71e-552802e97830 None admin] RBAC: Adding query filter params (role.id=9fe465cfe8764112a4cc92f3480326b8, scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74, user.id=54d0a424442447c49a385a116cc9eec2) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:44.510468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166b694-da08-4689-a71e-552802e97830 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:44.511976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166b694-da08-4689-a71e-552802e97830 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:44.518715 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 47/90] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:31:44 2022] GET /identity/v3/role_assignments?user.id=54d0a424442447c49a385a116cc9eec2&scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74&role.id=9fe465cfe8764112a4cc92f3480326b8 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:44.518879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 44/91] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:31:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:44.518957 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166b694-da08-4689-a71e-552802e97830 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:44.519243 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df08903-6d27-48d9-b0c4-51fc787899dc None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:44.568123 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:44.639412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166b694-da08-4689-a71e-552802e97830 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:44.642794 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df08903-6d27-48d9-b0c4-51fc787899dc None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:44.689975 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d9eb4-dcb4-4b07-b9a6-4a07f3d1bf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:44.690612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d9eb4-dcb4-4b07-b9a6-4a07f3d1bf2e None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:44.690896 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d9eb4-dcb4-4b07-b9a6-4a07f3d1bf2e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:44.691174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d9eb4-dcb4-4b07-b9a6-4a07f3d1bf2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:44.691930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d9eb4-dcb4-4b07-b9a6-4a07f3d1bf2e None admin] RBAC: Adding query filter params (role.id=8183d3986255476f959797314695f92a, scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74, user.id=54d0a424442447c49a385a116cc9eec2) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:44.692245 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d9eb4-dcb4-4b07-b9a6-4a07f3d1bf2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:44.693809 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d9eb4-dcb4-4b07-b9a6-4a07f3d1bf2e None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:44.696677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83b8d6-3d7c-43a7-9bd1-70a78f8bc9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:44.697323 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83b8d6-3d7c-43a7-9bd1-70a78f8bc9d4 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:44.697604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83b8d6-3d7c-43a7-9bd1-70a78f8bc9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:44.697881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83b8d6-3d7c-43a7-9bd1-70a78f8bc9d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:44.698631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83b8d6-3d7c-43a7-9bd1-70a78f8bc9d4 None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74, user.id=7275460c8c2e486f8768d1910d8aaa60) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:44.698946 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83b8d6-3d7c-43a7-9bd1-70a78f8bc9d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:44.700433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83b8d6-3d7c-43a7-9bd1-70a78f8bc9d4 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:44.713059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 45/92] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:31:44 2022] GET /identity/v3/role_assignments?user.id=54d0a424442447c49a385a116cc9eec2&scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74&role.id=8183d3986255476f959797314695f92a => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:44.713934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d9eb4-dcb4-4b07-b9a6-4a07f3d1bf2e None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:44.714567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 48/93] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:31:44 2022] GET /identity/v3/role_assignments?user.id=7275460c8c2e486f8768d1910d8aaa60&scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:44.856555 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:44.860825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d9eb4-dcb4-4b07-b9a6-4a07f3d1bf2e None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:44.905714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb0cb4-ca21-43ac-a68d-da4fdba89511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:44.906375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb0cb4-ca21-43ac-a68d-da4fdba89511 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:44.906660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb0cb4-ca21-43ac-a68d-da4fdba89511 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:44.906940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb0cb4-ca21-43ac-a68d-da4fdba89511 None admin] PATH_INFO: `/v3/users/54d0a424442447c49a385a116cc9eec2` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:44.908108 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb0cb4-ca21-43ac-a68d-da4fdba89511 None admin] RBAC: Authorizing `identity:get_user(user_id=54d0a424442447c49a385a116cc9eec2)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:44.909590 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb0cb4-ca21-43ac-a68d-da4fdba89511 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:44.911179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 46/94] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:44 2022] GET /identity/v3/users/54d0a424442447c49a385a116cc9eec2 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:44.917113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83b8d6-3d7c-43a7-9bd1-70a78f8bc9d4 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:44.957509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:45.088340 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83b8d6-3d7c-43a7-9bd1-70a78f8bc9d4 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:45.137032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff9a6b-b9d1-4565-a5f8-48175798eec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:45.137579 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff9a6b-b9d1-4565-a5f8-48175798eec9 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:45.137868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff9a6b-b9d1-4565-a5f8-48175798eec9 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:45.138149 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff9a6b-b9d1-4565-a5f8-48175798eec9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:45.227297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff9a6b-b9d1-4565-a5f8-48175798eec9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:45.228810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff9a6b-b9d1-4565-a5f8-48175798eec9 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:45.277108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:45.341059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 49/95] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:31:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:31:45.350449 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb0cb4-ca21-43ac-a68d-da4fdba89511 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:45.475253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb0cb4-ca21-43ac-a68d-da4fdba89511 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:45.512160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f39b4-b683-437a-ad17-d6cd96d2f6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:45.512900 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f39b4-b683-437a-ad17-d6cd96d2f6f8 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:45.513183 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f39b4-b683-437a-ad17-d6cd96d2f6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:45.513458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f39b4-b683-437a-ad17-d6cd96d2f6f8 None admin] PATH_INFO: `/v3/projects/772bf126b9df4caaac87f3a14bfd2d32` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:45.514663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f39b4-b683-437a-ad17-d6cd96d2f6f8 None admin] RBAC: Authorizing `identity:get_project(project_id=772bf126b9df4caaac87f3a14bfd2d32)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:45.516114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f39b4-b683-437a-ad17-d6cd96d2f6f8 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:45.518820 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 47/96] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:45 2022] GET /identity/v3/projects/772bf126b9df4caaac87f3a14bfd2d32 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:45.524233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff9a6b-b9d1-4565-a5f8-48175798eec9 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:45.676815 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff9a6b-b9d1-4565-a5f8-48175798eec9 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:45.719988 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:45.725191 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926af14b-1ad9-45be-b78b-30e022a4062b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:45.725889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926af14b-1ad9-45be-b78b-30e022a4062b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:45.726184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926af14b-1ad9-45be-b78b-30e022a4062b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:45.726471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926af14b-1ad9-45be-b78b-30e022a4062b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:45.727207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926af14b-1ad9-45be-b78b-30e022a4062b None admin] RBAC: Adding query filter params (role.id=9fe465cfe8764112a4cc92f3480326b8, scope.project.id=772bf126b9df4caaac87f3a14bfd2d32, user.id=54d0a424442447c49a385a116cc9eec2) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:45.727525 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926af14b-1ad9-45be-b78b-30e022a4062b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:45.728993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926af14b-1ad9-45be-b78b-30e022a4062b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:45.741011 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 50/97] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:31:45 2022] GET /identity/v3/role_assignments?user.id=54d0a424442447c49a385a116cc9eec2&scope.project.id=772bf126b9df4caaac87f3a14bfd2d32&role.id=9fe465cfe8764112a4cc92f3480326b8 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:46.893870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88b4df-cd2b-4b70-be40-bbf916f0243f None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:46.894412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88b4df-cd2b-4b70-be40-bbf916f0243f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:46.894412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88b4df-cd2b-4b70-be40-bbf916f0243f None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:46.896014 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 48/98] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:46.902112 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f70a79-6a89-467c-ab5a-1b2c9f99bb32 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:46.902523 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f70a79-6a89-467c-ab5a-1b2c9f99bb32 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:46.902668 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f70a79-6a89-467c-ab5a-1b2c9f99bb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:46.929763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-15f70a79-6a89-467c-ab5a-1b2c9f99bb32 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:46.931190 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-15f70a79-6a89-467c-ab5a-1b2c9f99bb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:47.019370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 51/99] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:47.368946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df33a0-77ad-4960-9705-fc12fa8af8f5 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:47.369592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df33a0-77ad-4960-9705-fc12fa8af8f5 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:47.370009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df33a0-77ad-4960-9705-fc12fa8af8f5 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:47.371481 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 49/100] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:47.386479 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f77f9-a42a-4d28-b491-fd39c5f4ad0f None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:47.386958 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f77f9-a42a-4d28-b491-fd39c5f4ad0f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:47.387406 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f77f9-a42a-4d28-b491-fd39c5f4ad0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:47.410321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-261f77f9-a42a-4d28-b491-fd39c5f4ad0f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:47.411749 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-261f77f9-a42a-4d28-b491-fd39c5f4ad0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:47.500668 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 52/101] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:47.543770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df33a0-77ad-4960-9705-fc12fa8af8f5 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:47.591690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:47.676512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df33a0-77ad-4960-9705-fc12fa8af8f5 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:47.719926 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f2c94-96cf-4efb-b0b6-aa866348f022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:47.720897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f2c94-96cf-4efb-b0b6-aa866348f022 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:47.721317 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f2c94-96cf-4efb-b0b6-aa866348f022 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:47.721748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f2c94-96cf-4efb-b0b6-aa866348f022 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:47.723098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f2c94-96cf-4efb-b0b6-aa866348f022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:47.724281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934ef6a-26c7-455f-82a3-5b1e4b5d63d6 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:47.724836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934ef6a-26c7-455f-82a3-5b1e4b5d63d6 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:47.725230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934ef6a-26c7-455f-82a3-5b1e4b5d63d6 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:47.725675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f2c94-96cf-4efb-b0b6-aa866348f022 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:47.726805 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 53/102] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:31:47.727626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 50/103] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:47.731998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21ea32-c55b-433b-942b-5d3f5467ac04 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:47.732401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21ea32-c55b-433b-942b-5d3f5467ac04 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:47.732828 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21ea32-c55b-433b-942b-5d3f5467ac04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:47.733354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f2c94-96cf-4efb-b0b6-aa866348f022 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:47.763337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-3b21ea32-c55b-433b-942b-5d3f5467ac04 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:47.764734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3b21ea32-c55b-433b-942b-5d3f5467ac04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:47.851102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:47.869103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 54/104] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:47.870763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fc41c-54e6-4e79-92b8-7383496f6320 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:47.871197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fc41c-54e6-4e79-92b8-7383496f6320 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:47.871622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fc41c-54e6-4e79-92b8-7383496f6320 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:47.873058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 55/105] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:47.876764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f2c94-96cf-4efb-b0b6-aa866348f022 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:47.877739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fc41c-54e6-4e79-92b8-7383496f6320 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:47.881701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:47.936553 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b0221-19f6-4ea7-9b45-39675c03dde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:47.937189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b0221-19f6-4ea7-9b45-39675c03dde5 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:47.937921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b0221-19f6-4ea7-9b45-39675c03dde5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:47.938204 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b0221-19f6-4ea7-9b45-39675c03dde5 None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:47.939146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b0221-19f6-4ea7-9b45-39675c03dde5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:47.940993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b0221-19f6-4ea7-9b45-39675c03dde5 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:47.992097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fc41c-54e6-4e79-92b8-7383496f6320 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:48.018041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 51/106] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 24 07:31:47 2022] POST /identity/v3/projects => generated 302 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:31:48.019701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b82af-16d1-4630-ad59-902801f092de None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:48.020333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b82af-16d1-4630-ad59-902801f092de None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:48.021485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b82af-16d1-4630-ad59-902801f092de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:48.038770 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873612f-17fb-4467-b036-e5fd95b36010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:48.039636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873612f-17fb-4467-b036-e5fd95b36010 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:48.039997 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873612f-17fb-4467-b036-e5fd95b36010 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:48.040399 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873612f-17fb-4467-b036-e5fd95b36010 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:48.049482 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-425b82af-16d1-4630-ad59-902801f092de None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:48.051557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-425b82af-16d1-4630-ad59-902801f092de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:48.053925 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873612f-17fb-4467-b036-e5fd95b36010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-0873612f-17fb-4467-b036-e5fd95b36010 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:48.059739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:48.060591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:48.061378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:48.061378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:48.061378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:48.061378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:48.061378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:48.061378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:48.061378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:48.061378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:48.061378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:48.061378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:48.061378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:31:48.061378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:31:48.061378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 56/107] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:48.062219 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb377-5874-4f0a-9a18-503f6f15a92c None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:48.062884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb377-5874-4f0a-9a18-503f6f15a92c None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:48.063319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb377-5874-4f0a-9a18-503f6f15a92c None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:48.066205 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 57/108] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:48.066478 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fb377-5874-4f0a-9a18-503f6f15a92c None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:48.125785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:48.163351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 52/109] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:48.165180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb14844-6716-4096-a0e5-a3f0959c5d01 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:48.165657 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb14844-6716-4096-a0e5-a3f0959c5d01 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:48.166136 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb14844-6716-4096-a0e5-a3f0959c5d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:48.188828 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-9cb14844-6716-4096-a0e5-a3f0959c5d01 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:48.190897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9cb14844-6716-4096-a0e5-a3f0959c5d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:48.233382 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fb377-5874-4f0a-9a18-503f6f15a92c None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:48.275817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 53/110] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:48.297894 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb14844-6716-4096-a0e5-a3f0959c5d01 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:48.317104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:48.335186 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e58b9d-5827-4875-b7bf-35cf649bdaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:48.335821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58b9d-5827-4875-b7bf-35cf649bdaa9 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:48.336109 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58b9d-5827-4875-b7bf-35cf649bdaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:48.336391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58b9d-5827-4875-b7bf-35cf649bdaa9 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:48.337110 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e58b9d-5827-4875-b7bf-35cf649bdaa9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:48.337426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e58b9d-5827-4875-b7bf-35cf649bdaa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:48.339514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e58b9d-5827-4875-b7bf-35cf649bdaa9 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:48.350183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 58/111] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:31:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:48.352200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644de2d-f247-4832-a342-53473c0ae07b None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:48.353470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644de2d-f247-4832-a342-53473c0ae07b None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:48.354300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644de2d-f247-4832-a342-53473c0ae07b None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:48.356405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 59/112] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:48.357604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644de2d-f247-4832-a342-53473c0ae07b None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:48.421976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb14844-6716-4096-a0e5-a3f0959c5d01 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:48.460323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d50ce-88e1-4444-a634-9e52563b0b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:48.461115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d50ce-88e1-4444-a634-9e52563b0b92 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:48.461432 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d50ce-88e1-4444-a634-9e52563b0b92 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:48.461784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d50ce-88e1-4444-a634-9e52563b0b92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:48.473049 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d50ce-88e1-4444-a634-9e52563b0b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-4b6d50ce-88e1-4444-a634-9e52563b0b92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:48.479105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:48.479946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:48.480731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:48.480731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:48.480731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:48.480731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:48.480731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:48.480731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:48.480731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:48.480731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:48.480731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:48.480731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:48.480731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:31:48.480731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:31:48.480731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 54/113] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:31:48.482638 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45955f2-94e8-4991-8892-f5bfba6f1f5d None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:48.482928 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45955f2-94e8-4991-8892-f5bfba6f1f5d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:48.483208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45955f2-94e8-4991-8892-f5bfba6f1f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:48.493726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644de2d-f247-4832-a342-53473c0ae07b None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:48.502241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:48.503432 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-a45955f2-94e8-4991-8892-f5bfba6f1f5d None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:48.504705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-a45955f2-94e8-4991-8892-f5bfba6f1f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:48.531714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854da84f-0f12-4d42-b9dc-7fc474166040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:48.532241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854da84f-0f12-4d42-b9dc-7fc474166040 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:48.532532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854da84f-0f12-4d42-b9dc-7fc474166040 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:48.532874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854da84f-0f12-4d42-b9dc-7fc474166040 None admin] PATH_INFO: `/v3/users/d3115fa7537242488412fff3edfdebce` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:48.550733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854da84f-0f12-4d42-b9dc-7fc474166040 None admin] RBAC: Authorizing `identity:get_user(user_id=d3115fa7537242488412fff3edfdebce)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:48.552204 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854da84f-0f12-4d42-b9dc-7fc474166040 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:48.554519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 60/114] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:48 2022] GET /identity/v3/users/d3115fa7537242488412fff3edfdebce => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:48.555303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854da84f-0f12-4d42-b9dc-7fc474166040 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:48.595316 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 55/115] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:48.602681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45955f2-94e8-4991-8892-f5bfba6f1f5d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:48.606878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:48.689959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854da84f-0f12-4d42-b9dc-7fc474166040 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:48.730608 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45955f2-94e8-4991-8892-f5bfba6f1f5d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:48.737798 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882ac4b-3750-41a2-ae4e-27e94fde891a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:48.738789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882ac4b-3750-41a2-ae4e-27e94fde891a None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:48.739272 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882ac4b-3750-41a2-ae4e-27e94fde891a None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:48.739745 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882ac4b-3750-41a2-ae4e-27e94fde891a None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:48.741251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882ac4b-3750-41a2-ae4e-27e94fde891a None admin] RBAC: Adding query filter params (name=member) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:48.741856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882ac4b-3750-41a2-ae4e-27e94fde891a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:48.743509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882ac4b-3750-41a2-ae4e-27e94fde891a None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:48.752303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 61/116] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:31:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:48.753475 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882ac4b-3750-41a2-ae4e-27e94fde891a None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:48.772895 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e991b-aeec-4c7f-b404-98f923bb717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:48.773550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e991b-aeec-4c7f-b404-98f923bb717a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:48.773835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e991b-aeec-4c7f-b404-98f923bb717a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:48.774111 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e991b-aeec-4c7f-b404-98f923bb717a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:48.784258 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:48.787838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e991b-aeec-4c7f-b404-98f923bb717a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-a52e991b-aeec-4c7f-b404-98f923bb717a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:48.793562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:48.800127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:48.801116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:48.801116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:48.801116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:48.801116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:48.801116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:48.801116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:48.801116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:48.801116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:48.801116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:48.801116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:48.801116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:48.801116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:31:48.801116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 56/117] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:31:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:48.801116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e991b-aeec-4c7f-b404-98f923bb717a None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:48.862518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882ac4b-3750-41a2-ae4e-27e94fde891a None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:48.866298 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:48.900677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cfae0-027c-4317-ad63-307b8e48ec41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:48.901525 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cfae0-027c-4317-ad63-307b8e48ec41 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:48.901953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cfae0-027c-4317-ad63-307b8e48ec41 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:48.902370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cfae0-027c-4317-ad63-307b8e48ec41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:48.910714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e991b-aeec-4c7f-b404-98f923bb717a None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:48.946545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c35ede-9924-4301-9c3d-ad6e4d319b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:48.947184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c35ede-9924-4301-9c3d-ad6e4d319b7e None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:48.947475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c35ede-9924-4301-9c3d-ad6e4d319b7e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:48.947758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c35ede-9924-4301-9c3d-ad6e4d319b7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:48.958689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c35ede-9924-4301-9c3d-ad6e4d319b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-19c35ede-9924-4301-9c3d-ad6e4d319b7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:48.965300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:48.966354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:48.967133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:48.967133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:48.967133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:48.967133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:48.967133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:48.967133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:48.967133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:48.967133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:48.967133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:48.967133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:48.967133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:31:48.967133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:31:48.967133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 57/118] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:31:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:48.967748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c35ede-9924-4301-9c3d-ad6e4d319b7e None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:48.984382 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cfae0-027c-4317-ad63-307b8e48ec41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:48.985816 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cfae0-027c-4317-ad63-307b8e48ec41 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:49.015465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:49.090849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 62/119] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:31:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:31:49.090849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cfae0-027c-4317-ad63-307b8e48ec41 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:49.113384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c35ede-9924-4301-9c3d-ad6e4d319b7e None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:49.139588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:49.150303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd093e46-9d92-4b98-925d-ab4e3fc5790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:49.150950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd093e46-9d92-4b98-925d-ab4e3fc5790e None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:49.151222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd093e46-9d92-4b98-925d-ab4e3fc5790e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:49.151501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd093e46-9d92-4b98-925d-ab4e3fc5790e None admin] PATH_INFO: `/v3/users/54d0a424442447c49a385a116cc9eec2` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:49.153309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd093e46-9d92-4b98-925d-ab4e3fc5790e None admin] RBAC: Authorizing `identity:get_user(user_id=54d0a424442447c49a385a116cc9eec2)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:49.154118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd093e46-9d92-4b98-925d-ab4e3fc5790e None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:49.155691 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 58/120] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:48 2022] GET /identity/v3/users/54d0a424442447c49a385a116cc9eec2 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:49.156450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd093e46-9d92-4b98-925d-ab4e3fc5790e None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:49.198554 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cfae0-027c-4317-ad63-307b8e48ec41 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:49.227345 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:49.240712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dbe77-9ef9-4ee2-9bae-93f8e76910c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:49.240712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dbe77-9ef9-4ee2-9bae-93f8e76910c3 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:49.240712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dbe77-9ef9-4ee2-9bae-93f8e76910c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:49.240712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dbe77-9ef9-4ee2-9bae-93f8e76910c3 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:49.241078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3dbe77-9ef9-4ee2-9bae-93f8e76910c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:49.241078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3dbe77-9ef9-4ee2-9bae-93f8e76910c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:49.241078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3dbe77-9ef9-4ee2-9bae-93f8e76910c3 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:49.250022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 63/121] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:49.255431 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dbe77-9ef9-4ee2-9bae-93f8e76910c3 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:49.270901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd093e46-9d92-4b98-925d-ab4e3fc5790e None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:49.316041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64384158-a6ad-4c5e-9aaa-9fabd1b6abeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:49.316904 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64384158-a6ad-4c5e-9aaa-9fabd1b6abeb None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:49.317192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64384158-a6ad-4c5e-9aaa-9fabd1b6abeb None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:49.317583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64384158-a6ad-4c5e-9aaa-9fabd1b6abeb None admin] PATH_INFO: `/v3/roles/member` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:49.330725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64384158-a6ad-4c5e-9aaa-9fabd1b6abeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-64384158-a6ad-4c5e-9aaa-9fabd1b6abeb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:49.337114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:49.337985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:49.338735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:49.338735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:49.338735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:49.338735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:49.338735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:49.338735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:49.338735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:49.338735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:49.338735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:49.338735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:49.338735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:31:49.338735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:31:49.338735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 59/122] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:49.339356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64384158-a6ad-4c5e-9aaa-9fabd1b6abeb None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:49.364229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dbe77-9ef9-4ee2-9bae-93f8e76910c3 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:49.394774 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:49.401681 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b6491-3c28-40c8-ba09-fa981ca3acee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:49.402300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b6491-3c28-40c8-ba09-fa981ca3acee None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:49.402589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b6491-3c28-40c8-ba09-fa981ca3acee None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:49.402862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b6491-3c28-40c8-ba09-fa981ca3acee None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:49.403531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b6491-3c28-40c8-ba09-fa981ca3acee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:49.403839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b6491-3c28-40c8-ba09-fa981ca3acee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:49.405341 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b6491-3c28-40c8-ba09-fa981ca3acee None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:49.418135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 64/123] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:31:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:49.418135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b6491-3c28-40c8-ba09-fa981ca3acee None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:49.446528 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64384158-a6ad-4c5e-9aaa-9fabd1b6abeb None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:49.476387 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:49.483015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b187c4b-2af3-47d2-b2d5-cb3e9742a5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:49.483710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b187c4b-2af3-47d2-b2d5-cb3e9742a5be None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:49.483993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b187c4b-2af3-47d2-b2d5-cb3e9742a5be None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:49.484271 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b187c4b-2af3-47d2-b2d5-cb3e9742a5be None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:49.485563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b187c4b-2af3-47d2-b2d5-cb3e9742a5be None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:49.487024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b187c4b-2af3-47d2-b2d5-cb3e9742a5be None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:49.491028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 60/124] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:49 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:49.493654 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b187c4b-2af3-47d2-b2d5-cb3e9742a5be None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:49.524142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b6491-3c28-40c8-ba09-fa981ca3acee None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:49.562668 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee303b4-4047-4962-95ed-b27391187cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:49.563307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee303b4-4047-4962-95ed-b27391187cae None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:49.563596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee303b4-4047-4962-95ed-b27391187cae None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:49.563878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee303b4-4047-4962-95ed-b27391187cae None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:49.565097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee303b4-4047-4962-95ed-b27391187cae None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:49.566561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee303b4-4047-4962-95ed-b27391187cae None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:49.573807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 65/125] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:49 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:49.573807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee303b4-4047-4962-95ed-b27391187cae None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:49.621708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b187c4b-2af3-47d2-b2d5-cb3e9742a5be None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:49.642388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:49.656874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2238c-24b1-4a29-9535-1e7222248862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:49.659333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2238c-24b1-4a29-9535-1e7222248862 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:49.659333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2238c-24b1-4a29-9535-1e7222248862 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:49.659333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2238c-24b1-4a29-9535-1e7222248862 None admin] PATH_INFO: `/v3/users/7275460c8c2e486f8768d1910d8aaa60` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:49.659333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2238c-24b1-4a29-9535-1e7222248862 None admin] RBAC: Authorizing `identity:get_user(user_id=7275460c8c2e486f8768d1910d8aaa60)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:49.660753 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2238c-24b1-4a29-9535-1e7222248862 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:49.662713 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 61/126] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:49 2022] GET /identity/v3/users/7275460c8c2e486f8768d1910d8aaa60 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:49.667723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2238c-24b1-4a29-9535-1e7222248862 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:49.689727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee303b4-4047-4962-95ed-b27391187cae None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:49.711022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:49.725912 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20beee8-4797-452b-a5c0-53e3f341063e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:49.726673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20beee8-4797-452b-a5c0-53e3f341063e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:49.727071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20beee8-4797-452b-a5c0-53e3f341063e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:49.727450 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20beee8-4797-452b-a5c0-53e3f341063e None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:49.728127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20beee8-4797-452b-a5c0-53e3f341063e None admin] RBAC: Adding query filter params (name=member) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:49.728437 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20beee8-4797-452b-a5c0-53e3f341063e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:49.729975 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20beee8-4797-452b-a5c0-53e3f341063e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:49.741844 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 66/127] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:31:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:49.743135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20beee8-4797-452b-a5c0-53e3f341063e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:49.785991 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2238c-24b1-4a29-9535-1e7222248862 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:49.821520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c86c6-ccff-4272-96f8-102057a71a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:49.822156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c86c6-ccff-4272-96f8-102057a71a49 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:49.822437 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c86c6-ccff-4272-96f8-102057a71a49 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:49.822715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c86c6-ccff-4272-96f8-102057a71a49 None admin] PATH_INFO: `/v3/users/54d0a424442447c49a385a116cc9eec2` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:49.823835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668c86c6-ccff-4272-96f8-102057a71a49 None admin] RBAC: Authorizing `identity:get_user(user_id=54d0a424442447c49a385a116cc9eec2)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:49.825324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668c86c6-ccff-4272-96f8-102057a71a49 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:49.826884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 62/128] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:49 2022] GET /identity/v3/users/54d0a424442447c49a385a116cc9eec2 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:49.827624 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c86c6-ccff-4272-96f8-102057a71a49 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:49.858252 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20beee8-4797-452b-a5c0-53e3f341063e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:49.887147 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:49.895437 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef373d-0c72-4603-aafc-e04087432525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:49.898511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef373d-0c72-4603-aafc-e04087432525 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:49.898511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef373d-0c72-4603-aafc-e04087432525 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:49.898511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef373d-0c72-4603-aafc-e04087432525 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:49.898916 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef373d-0c72-4603-aafc-e04087432525 None admin] RBAC: Adding query filter params (role.id=27470a2fbc754a179323bb892496b013, scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74, user.id=d3115fa7537242488412fff3edfdebce) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:49.899376 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef373d-0c72-4603-aafc-e04087432525 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:49.901025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef373d-0c72-4603-aafc-e04087432525 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:49.907129 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 67/129] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:31:49 2022] GET /identity/v3/role_assignments?user.id=d3115fa7537242488412fff3edfdebce&scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74&role.id=27470a2fbc754a179323bb892496b013 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:49.908155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef373d-0c72-4603-aafc-e04087432525 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:49.945903 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c86c6-ccff-4272-96f8-102057a71a49 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:49.969873 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:49.989625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b387a7-3e1f-4e77-b0df-424292fbe4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:49.990851 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b387a7-3e1f-4e77-b0df-424292fbe4ea None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:49.991224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b387a7-3e1f-4e77-b0df-424292fbe4ea None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:49.991518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b387a7-3e1f-4e77-b0df-424292fbe4ea None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74/users/54d0a424442447c49a385a116cc9eec2/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:49.994785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b387a7-3e1f-4e77-b0df-424292fbe4ea None admin] RBAC: Authorizing `identity:create_grant(project_id=0c39a23af4154a8b9ada3f37e9fa6e74, user_id=54d0a424442447c49a385a116cc9eec2, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:49.996563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b387a7-3e1f-4e77-b0df-424292fbe4ea None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:50.038937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef373d-0c72-4603-aafc-e04087432525 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:50.072295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 63/130] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:31:49 2022] PUT /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74/users/54d0a424442447c49a385a116cc9eec2/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:31:50.073472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b387a7-3e1f-4e77-b0df-424292fbe4ea None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:50.080378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe721c5b-e5fb-4499-97b7-c4e2846494c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:50.081171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe721c5b-e5fb-4499-97b7-c4e2846494c5 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:50.081539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe721c5b-e5fb-4499-97b7-c4e2846494c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:50.081882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe721c5b-e5fb-4499-97b7-c4e2846494c5 None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:50.083184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe721c5b-e5fb-4499-97b7-c4e2846494c5 None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:50.085619 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe721c5b-e5fb-4499-97b7-c4e2846494c5 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:50.087595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 68/131] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:49 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:50.088440 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe721c5b-e5fb-4499-97b7-c4e2846494c5 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:50.220341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b387a7-3e1f-4e77-b0df-424292fbe4ea None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:50.242117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:50.265308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1f7a9-6869-4d68-8bd9-a0ff6c61af89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:50.266462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1f7a9-6869-4d68-8bd9-a0ff6c61af89 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:50.266886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1f7a9-6869-4d68-8bd9-a0ff6c61af89 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:50.267301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1f7a9-6869-4d68-8bd9-a0ff6c61af89 None admin] PATH_INFO: `/v3/users/54d0a424442447c49a385a116cc9eec2` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:50.267757 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe721c5b-e5fb-4499-97b7-c4e2846494c5 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:50.268826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1f7a9-6869-4d68-8bd9-a0ff6c61af89 None admin] RBAC: Authorizing `identity:get_user(user_id=54d0a424442447c49a385a116cc9eec2)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:50.270743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1f7a9-6869-4d68-8bd9-a0ff6c61af89 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:50.272844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 64/132] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:50 2022] GET /identity/v3/users/54d0a424442447c49a385a116cc9eec2 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:50.274970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1f7a9-6869-4d68-8bd9-a0ff6c61af89 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:50.304549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:50.329465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc63f61-ac42-41cd-9a95-850c824b0c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:50.330535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc63f61-ac42-41cd-9a95-850c824b0c12 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:50.330809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc63f61-ac42-41cd-9a95-850c824b0c12 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:50.331088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc63f61-ac42-41cd-9a95-850c824b0c12 None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:50.332224 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc63f61-ac42-41cd-9a95-850c824b0c12 None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:50.333787 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc63f61-ac42-41cd-9a95-850c824b0c12 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:50.335605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 69/133] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:50 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:50.336384 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc63f61-ac42-41cd-9a95-850c824b0c12 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:50.410091 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1f7a9-6869-4d68-8bd9-a0ff6c61af89 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:50.447866 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dca954-dba9-4eaa-a670-259250da5edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:50.448781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dca954-dba9-4eaa-a670-259250da5edc None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:50.449185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dca954-dba9-4eaa-a670-259250da5edc None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:50.449609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dca954-dba9-4eaa-a670-259250da5edc None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74/users/7275460c8c2e486f8768d1910d8aaa60/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:50.451647 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dca954-dba9-4eaa-a670-259250da5edc None admin] RBAC: Authorizing `identity:create_grant(project_id=0c39a23af4154a8b9ada3f37e9fa6e74, user_id=7275460c8c2e486f8768d1910d8aaa60, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:50.453245 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dca954-dba9-4eaa-a670-259250da5edc None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:50.459734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc63f61-ac42-41cd-9a95-850c824b0c12 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:50.494749 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d4696-513c-4873-87b2-2102ba38826b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:50.495459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d4696-513c-4873-87b2-2102ba38826b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:50.495739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d4696-513c-4873-87b2-2102ba38826b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:50.496012 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d4696-513c-4873-87b2-2102ba38826b None admin] PATH_INFO: `/v3/projects/772bf126b9df4caaac87f3a14bfd2d32` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:50.497176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d4696-513c-4873-87b2-2102ba38826b None admin] RBAC: Authorizing `identity:get_project(project_id=772bf126b9df4caaac87f3a14bfd2d32)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:50.498740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d4696-513c-4873-87b2-2102ba38826b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:50.500622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 70/134] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:50 2022] GET /identity/v3/projects/772bf126b9df4caaac87f3a14bfd2d32 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:50.501561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640479a-78ad-469c-8de6-0d5398ee2c07 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:50.501844 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640479a-78ad-469c-8de6-0d5398ee2c07 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:50.502120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640479a-78ad-469c-8de6-0d5398ee2c07 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:50.503301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 71/135] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:50.508215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640479a-78ad-469c-8de6-0d5398ee2c07 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:50.555299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 65/136] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:31:50 2022] PUT /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74/users/7275460c8c2e486f8768d1910d8aaa60/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:31:50.564675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dca954-dba9-4eaa-a670-259250da5edc None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:50.589935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:50.667742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:50.706701 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640479a-78ad-469c-8de6-0d5398ee2c07 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:50.752568 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2848e9d-a9f7-4262-90d9-82ad14f12b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:50.753294 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2848e9d-a9f7-4262-90d9-82ad14f12b53 None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:50.753588 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2848e9d-a9f7-4262-90d9-82ad14f12b53 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:50.753871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2848e9d-a9f7-4262-90d9-82ad14f12b53 None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74/users/54d0a424442447c49a385a116cc9eec2/roles/8183d3986255476f959797314695f92a` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:50.757180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2848e9d-a9f7-4262-90d9-82ad14f12b53 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c39a23af4154a8b9ada3f37e9fa6e74, user_id=54d0a424442447c49a385a116cc9eec2, role_id=8183d3986255476f959797314695f92a)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:50.757757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dca954-dba9-4eaa-a670-259250da5edc None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:50.758652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2848e9d-a9f7-4262-90d9-82ad14f12b53 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:50.801879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fba52c-be9c-4ecd-b833-8ee875ca04e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:50.802633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fba52c-be9c-4ecd-b833-8ee875ca04e5 None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:50.802991 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fba52c-be9c-4ecd-b833-8ee875ca04e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:50.803418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fba52c-be9c-4ecd-b833-8ee875ca04e5 None admin] PATH_INFO: `/v3/projects/772bf126b9df4caaac87f3a14bfd2d32/users/54d0a424442447c49a385a116cc9eec2/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:50.805445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fba52c-be9c-4ecd-b833-8ee875ca04e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=772bf126b9df4caaac87f3a14bfd2d32, user_id=54d0a424442447c49a385a116cc9eec2, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:50.806985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fba52c-be9c-4ecd-b833-8ee875ca04e5 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:50.825441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 72/137] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:31:50 2022] PUT /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74/users/54d0a424442447c49a385a116cc9eec2/roles/8183d3986255476f959797314695f92a => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:31:50.827224 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9ac4-b59d-49e9-81bf-13ac65732aae None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:50.827723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9ac4-b59d-49e9-81bf-13ac65732aae None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:50.828158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9ac4-b59d-49e9-81bf-13ac65732aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:50.857170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-c2ac9ac4-b59d-49e9-81bf-13ac65732aae None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:50.858776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c2ac9ac4-b59d-49e9-81bf-13ac65732aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:50.897639 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 66/138] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:31:50 2022] PUT /identity/v3/projects/772bf126b9df4caaac87f3a14bfd2d32/users/54d0a424442447c49a385a116cc9eec2/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:31:50.955361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 73/139] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:51.301408 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fba52c-be9c-4ecd-b833-8ee875ca04e5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:51.413192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fba52c-be9c-4ecd-b833-8ee875ca04e5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:51.415559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:51.447912 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8ab6a-029c-4882-8a61-0e1bb52d99fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:51.448550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ab6a-029c-4882-8a61-0e1bb52d99fc None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:51.448863 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ab6a-029c-4882-8a61-0e1bb52d99fc None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:51.449139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ab6a-029c-4882-8a61-0e1bb52d99fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:51.450296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8ab6a-029c-4882-8a61-0e1bb52d99fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:51.451763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8ab6a-029c-4882-8a61-0e1bb52d99fc None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:51.453565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 67/140] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:51.460212 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac9ac4-b59d-49e9-81bf-13ac65732aae None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:51.503644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:51.565004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac9ac4-b59d-49e9-81bf-13ac65732aae None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:51.603858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f79dc8-ba48-4fd7-b69f-35b50e8913d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:51.604760 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f79dc8-ba48-4fd7-b69f-35b50e8913d1 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:51.605178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f79dc8-ba48-4fd7-b69f-35b50e8913d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:51.605603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f79dc8-ba48-4fd7-b69f-35b50e8913d1 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:51.606396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f79dc8-ba48-4fd7-b69f-35b50e8913d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:51.607979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f79dc8-ba48-4fd7-b69f-35b50e8913d1 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:51.722528 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 74/141] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:31:51 2022] POST /identity/v3/users => generated 283 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:31:52.258968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c71b37-2da3-41cd-8d41-1c57e16e6994 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:52.259674 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c71b37-2da3-41cd-8d41-1c57e16e6994 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:52.260170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c71b37-2da3-41cd-8d41-1c57e16e6994 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:52.261724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 68/142] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:52.267170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc0550-19c6-450c-a334-fd7062c308bd None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:52.267686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc0550-19c6-450c-a334-fd7062c308bd None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:52.268181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc0550-19c6-450c-a334-fd7062c308bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:52.288297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-45cc0550-19c6-450c-a334-fd7062c308bd None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:52.293919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-45cc0550-19c6-450c-a334-fd7062c308bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:52.374945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 75/143] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:52.654131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e953286-6ed0-480f-a392-f38f25c63023 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:52.655156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e953286-6ed0-480f-a392-f38f25c63023 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:52.655624 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e953286-6ed0-480f-a392-f38f25c63023 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:52.657427 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 69/144] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:52.664472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9e29d-897b-42c5-8fdc-32c4c0e4abdf None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:52.665373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9e29d-897b-42c5-8fdc-32c4c0e4abdf None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:52.665841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9e29d-897b-42c5-8fdc-32c4c0e4abdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:52.685937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-f7e9e29d-897b-42c5-8fdc-32c4c0e4abdf None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:52.687425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f7e9e29d-897b-42c5-8fdc-32c4c0e4abdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:52.708682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:52.758181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 76/145] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:52.780426 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e953286-6ed0-480f-a392-f38f25c63023 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:52.894219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e953286-6ed0-480f-a392-f38f25c63023 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:52.932029 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae25f4-10a4-40b9-acb6-f456e7704ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:52.932691 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae25f4-10a4-40b9-acb6-f456e7704ed5 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:52.932981 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae25f4-10a4-40b9-acb6-f456e7704ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:52.933255 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae25f4-10a4-40b9-acb6-f456e7704ed5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:52.944417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae25f4-10a4-40b9-acb6-f456e7704ed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-4cae25f4-10a4-40b9-acb6-f456e7704ed5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:52.949247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:52.950085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:52.950821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:52.950821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:52.950821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:52.950821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:52.950821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:52.950821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:52.950821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:52.950821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:52.950821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:52.950821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:52.950821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:31:52.950821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:31:52.950821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 70/146] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:52.956274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9e29d-897b-42c5-8fdc-32c4c0e4abdf None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:53.061726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9e29d-897b-42c5-8fdc-32c4c0e4abdf None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:53.084850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae25f4-10a4-40b9-acb6-f456e7704ed5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:53.124742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9245d7d-0d4d-42fb-b384-03654cfc1f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:53.125850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9245d7d-0d4d-42fb-b384-03654cfc1f91 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:53.126144 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9245d7d-0d4d-42fb-b384-03654cfc1f91 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:53.126425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9245d7d-0d4d-42fb-b384-03654cfc1f91 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:53.127117 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9245d7d-0d4d-42fb-b384-03654cfc1f91 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:53.127492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9245d7d-0d4d-42fb-b384-03654cfc1f91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:53.128982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9245d7d-0d4d-42fb-b384-03654cfc1f91 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:53.140992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 77/147] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:31:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:53.147476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9245d7d-0d4d-42fb-b384-03654cfc1f91 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:53.160032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:53.168998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:53.260173 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9245d7d-0d4d-42fb-b384-03654cfc1f91 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:53.268179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae25f4-10a4-40b9-acb6-f456e7704ed5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:53.297137 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dd8e8-8864-402d-8d28-86b645fbffea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:53.297862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dd8e8-8864-402d-8d28-86b645fbffea None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:53.298217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dd8e8-8864-402d-8d28-86b645fbffea None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:53.298571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dd8e8-8864-402d-8d28-86b645fbffea None admin] PATH_INFO: `/v3/users/d3115fa7537242488412fff3edfdebce` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:53.299853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dd8e8-8864-402d-8d28-86b645fbffea None admin] RBAC: Authorizing `identity:get_user(user_id=d3115fa7537242488412fff3edfdebce)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:53.301500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dd8e8-8864-402d-8d28-86b645fbffea None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:53.305798 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 78/148] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:53 2022] GET /identity/v3/users/d3115fa7537242488412fff3edfdebce => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:53.311485 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dd8e8-8864-402d-8d28-86b645fbffea None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:53.332487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d4620-baec-4ca7-a369-d488cb94a518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:53.333172 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d4620-baec-4ca7-a369-d488cb94a518 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:53.333459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d4620-baec-4ca7-a369-d488cb94a518 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:53.333874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d4620-baec-4ca7-a369-d488cb94a518 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:53.350073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d4620-baec-4ca7-a369-d488cb94a518 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-118d4620-baec-4ca7-a369-d488cb94a518 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:53.354742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:53.355575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:53.356333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:53.356333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:53.356333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:53.356333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:53.356333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:53.356333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:53.356333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:53.356333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:53.356333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:53.356333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:53.356333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:31:53.356333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:31:53.356333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 71/149] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:31:53.365946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d4620-baec-4ca7-a369-d488cb94a518 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:53.418266 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:53.426854 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dd8e8-8864-402d-8d28-86b645fbffea None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:53.490527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba95ac-0891-4783-b714-ed7d62be090e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:53.491341 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba95ac-0891-4783-b714-ed7d62be090e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:53.491627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba95ac-0891-4783-b714-ed7d62be090e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:53.491901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba95ac-0891-4783-b714-ed7d62be090e None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:53.494951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba95ac-0891-4783-b714-ed7d62be090e None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:53.496412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba95ac-0891-4783-b714-ed7d62be090e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:53.499644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 79/150] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:53 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:53.505523 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba95ac-0891-4783-b714-ed7d62be090e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:53.520137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:53.541473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d4620-baec-4ca7-a369-d488cb94a518 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:53.613228 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a62f1-5824-4552-9013-61ce88150d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:53.613995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a62f1-5824-4552-9013-61ce88150d3d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:53.614331 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a62f1-5824-4552-9013-61ce88150d3d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:53.614606 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a62f1-5824-4552-9013-61ce88150d3d None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:53.616150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a62f1-5824-4552-9013-61ce88150d3d None admin] RBAC: Adding query filter params (name=member) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:53.616508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a62f1-5824-4552-9013-61ce88150d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:53.618103 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a62f1-5824-4552-9013-61ce88150d3d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:53.622973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba95ac-0891-4783-b714-ed7d62be090e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:53.635339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 72/151] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:31:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:53.640989 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a62f1-5824-4552-9013-61ce88150d3d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:53.659963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f245cb-592c-43a0-9fe7-b41549a995c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:53.660689 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f245cb-592c-43a0-9fe7-b41549a995c8 None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:53.660979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f245cb-592c-43a0-9fe7-b41549a995c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:53.661301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f245cb-592c-43a0-9fe7-b41549a995c8 None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74/users/d3115fa7537242488412fff3edfdebce/roles/27470a2fbc754a179323bb892496b013` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:53.663449 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f245cb-592c-43a0-9fe7-b41549a995c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c39a23af4154a8b9ada3f37e9fa6e74, user_id=d3115fa7537242488412fff3edfdebce, role_id=27470a2fbc754a179323bb892496b013)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:53.665000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f245cb-592c-43a0-9fe7-b41549a995c8 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:53.755877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 80/152] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:31:53 2022] PUT /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74/users/d3115fa7537242488412fff3edfdebce/roles/27470a2fbc754a179323bb892496b013 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:31:53.799970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a62f1-5824-4552-9013-61ce88150d3d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:53.849614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03096215-3e45-49b0-9e32-e88306d30532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:53.850272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03096215-3e45-49b0-9e32-e88306d30532 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:53.850557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03096215-3e45-49b0-9e32-e88306d30532 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:53.850836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03096215-3e45-49b0-9e32-e88306d30532 None admin] PATH_INFO: `/v3/users/54d0a424442447c49a385a116cc9eec2` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:53.859656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03096215-3e45-49b0-9e32-e88306d30532 None admin] RBAC: Authorizing `identity:get_user(user_id=54d0a424442447c49a385a116cc9eec2)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:53.859814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb1632-4518-47ac-a74d-ac8529ee7722 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:53.859814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb1632-4518-47ac-a74d-ac8529ee7722 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:53.859814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb1632-4518-47ac-a74d-ac8529ee7722 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:53.859994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03096215-3e45-49b0-9e32-e88306d30532 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:53.860082 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 81/153] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:53.860547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 73/154] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:53 2022] GET /identity/v3/users/54d0a424442447c49a385a116cc9eec2 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:53.867079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03096215-3e45-49b0-9e32-e88306d30532 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:53.867353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344b69f-0b56-4857-a182-ae10558c07f9 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:53.867615 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344b69f-0b56-4857-a182-ae10558c07f9 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:53.867895 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344b69f-0b56-4857-a182-ae10558c07f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:53.908673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:53.908860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a344b69f-0b56-4857-a182-ae10558c07f9 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:53.908860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a344b69f-0b56-4857-a182-ae10558c07f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:53.933508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:54.032600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03096215-3e45-49b0-9e32-e88306d30532 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:54.034112 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 82/155] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:54.035434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfad3e6-81de-402b-9ea7-9096b4a2f29a None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:54.035721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfad3e6-81de-402b-9ea7-9096b4a2f29a None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:54.035992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfad3e6-81de-402b-9ea7-9096b4a2f29a None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:54.038264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 83/156] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:54.046153 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c6336-5ba9-4939-b62c-09752c105975 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:54.046474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c6336-5ba9-4939-b62c-09752c105975 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:54.046705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c6336-5ba9-4939-b62c-09752c105975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:54.088484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-994c6336-5ba9-4939-b62c-09752c105975 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:54.089872 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-994c6336-5ba9-4939-b62c-09752c105975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:54.100120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd30a5-1759-40f5-a89c-0756e6e41bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:54.101115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd30a5-1759-40f5-a89c-0756e6e41bd3 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:54.101695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd30a5-1759-40f5-a89c-0756e6e41bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:54.102166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd30a5-1759-40f5-a89c-0756e6e41bd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:54.199155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 84/157] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:54.227333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd30a5-1759-40f5-a89c-0756e6e41bd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:54.228861 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd30a5-1759-40f5-a89c-0756e6e41bd3 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:54.263570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:54.346488 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 74/158] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:31:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:31:54.352760 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c6336-5ba9-4939-b62c-09752c105975 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:54.453053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:54.483498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c6336-5ba9-4939-b62c-09752c105975 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:54.523190 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085cd93-8a6b-40fa-aac8-29d7fc661299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:54.524195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085cd93-8a6b-40fa-aac8-29d7fc661299 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:54.524742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085cd93-8a6b-40fa-aac8-29d7fc661299 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:54.525215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085cd93-8a6b-40fa-aac8-29d7fc661299 None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:54.526544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085cd93-8a6b-40fa-aac8-29d7fc661299 None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:54.528205 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085cd93-8a6b-40fa-aac8-29d7fc661299 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:54.530079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 85/159] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:54 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:54.536942 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd30a5-1759-40f5-a89c-0756e6e41bd3 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:54.646990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085cd93-8a6b-40fa-aac8-29d7fc661299 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:54.687911 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd30a5-1759-40f5-a89c-0756e6e41bd3 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:54.742950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:54.769391 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93eae13-a041-4cf6-a5ce-e21bafc370ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:54.770485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93eae13-a041-4cf6-a5ce-e21bafc370ef None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:54.771032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93eae13-a041-4cf6-a5ce-e21bafc370ef None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:54.772062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93eae13-a041-4cf6-a5ce-e21bafc370ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:54.773027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93eae13-a041-4cf6-a5ce-e21bafc370ef None admin] RBAC: Adding query filter params (role.id=9fe465cfe8764112a4cc92f3480326b8, scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74, user.id=54d0a424442447c49a385a116cc9eec2) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:54.773556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93eae13-a041-4cf6-a5ce-e21bafc370ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:54.775412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93eae13-a041-4cf6-a5ce-e21bafc370ef None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:54.787114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 75/160] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:31:54 2022] GET /identity/v3/role_assignments?user.id=54d0a424442447c49a385a116cc9eec2&scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74&role.id=9fe465cfe8764112a4cc92f3480326b8 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:54.788360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93eae13-a041-4cf6-a5ce-e21bafc370ef None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:54.803685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085cd93-8a6b-40fa-aac8-29d7fc661299 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:54.855661 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b261a-5078-4d9e-b617-72e67ca478fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:54.856557 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b261a-5078-4d9e-b617-72e67ca478fc None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:54.856993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b261a-5078-4d9e-b617-72e67ca478fc None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:54.857404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b261a-5078-4d9e-b617-72e67ca478fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:54.874322 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b261a-5078-4d9e-b617-72e67ca478fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-b79b261a-5078-4d9e-b617-72e67ca478fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:54.879716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:54.881315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:54.882179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:54.882179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:54.882179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:54.882179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:54.882179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:54.882179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:54.882179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:54.882179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:54.882179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:54.882179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:54.882179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:54.882179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:31:54.884612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 86/161] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:31:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:31:54.890679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b261a-5078-4d9e-b617-72e67ca478fc None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:54.894505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:55.010779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b261a-5078-4d9e-b617-72e67ca478fc None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:55.038870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93eae13-a041-4cf6-a5ce-e21bafc370ef None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:55.050469 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80560f-1bf4-4c6f-a8fc-c56495c0dcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:55.051478 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80560f-1bf4-4c6f-a8fc-c56495c0dcad None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:55.051927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80560f-1bf4-4c6f-a8fc-c56495c0dcad None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:55.052804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80560f-1bf4-4c6f-a8fc-c56495c0dcad None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:55.053832 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80560f-1bf4-4c6f-a8fc-c56495c0dcad None admin] RBAC: Adding query filter params (name=admin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:55.054313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80560f-1bf4-4c6f-a8fc-c56495c0dcad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:55.055933 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80560f-1bf4-4c6f-a8fc-c56495c0dcad None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:55.065556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 87/162] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:55.071248 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80560f-1bf4-4c6f-a8fc-c56495c0dcad None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:55.098447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229ec6b-310d-40ed-a3b2-22ac2c0aec35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:55.100302 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229ec6b-310d-40ed-a3b2-22ac2c0aec35 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:55.100976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229ec6b-310d-40ed-a3b2-22ac2c0aec35 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:55.101859 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229ec6b-310d-40ed-a3b2-22ac2c0aec35 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:55.118228 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229ec6b-310d-40ed-a3b2-22ac2c0aec35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-4229ec6b-310d-40ed-a3b2-22ac2c0aec35 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:55.125870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:55.126748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:55.127511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:55.127511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:55.127511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:55.127511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:55.127511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:55.127511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:55.127511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:55.127511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:55.127511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:55.127511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:55.127511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:31:55.127511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:31:55.129704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 76/163] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:31:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:55.136399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229ec6b-310d-40ed-a3b2-22ac2c0aec35 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:55.149883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:55.206369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80560f-1bf4-4c6f-a8fc-c56495c0dcad None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:55.212101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:55.252895 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309ba2e-b48a-49bf-b16e-816a958c688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:55.253536 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309ba2e-b48a-49bf-b16e-816a958c688b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:55.253817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309ba2e-b48a-49bf-b16e-816a958c688b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:55.254089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309ba2e-b48a-49bf-b16e-816a958c688b None admin] PATH_INFO: `/v3/users/7275460c8c2e486f8768d1910d8aaa60` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:55.255747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309ba2e-b48a-49bf-b16e-816a958c688b None admin] RBAC: Authorizing `identity:get_user(user_id=7275460c8c2e486f8768d1910d8aaa60)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:55.258405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309ba2e-b48a-49bf-b16e-816a958c688b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:55.264733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 88/164] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:55 2022] GET /identity/v3/users/7275460c8c2e486f8768d1910d8aaa60 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:55.264733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31adee-7e73-4867-8e1b-b2986debd675 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:55.264733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31adee-7e73-4867-8e1b-b2986debd675 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:55.264733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31adee-7e73-4867-8e1b-b2986debd675 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:55.264733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 89/165] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:55.267212 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31adee-7e73-4867-8e1b-b2986debd675 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:55.293139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229ec6b-310d-40ed-a3b2-22ac2c0aec35 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:55.352831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33309cd-59a0-4010-859f-5b0a7e38d8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:55.353482 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33309cd-59a0-4010-859f-5b0a7e38d8cb None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:55.353770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33309cd-59a0-4010-859f-5b0a7e38d8cb None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:55.354051 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33309cd-59a0-4010-859f-5b0a7e38d8cb None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:55.354701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33309cd-59a0-4010-859f-5b0a7e38d8cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:55.355004 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33309cd-59a0-4010-859f-5b0a7e38d8cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:55.356390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33309cd-59a0-4010-859f-5b0a7e38d8cb None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:55.369433 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 77/166] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:31:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:55.372241 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4c5c6-1ccb-4b91-8047-390b40aa6a83 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:55.372521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4c5c6-1ccb-4b91-8047-390b40aa6a83 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:55.378217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4c5c6-1ccb-4b91-8047-390b40aa6a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:55.413700 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-40f4c5c6-1ccb-4b91-8047-390b40aa6a83 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:55.415034 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-40f4c5c6-1ccb-4b91-8047-390b40aa6a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:55.457736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31adee-7e73-4867-8e1b-b2986debd675 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:55.517001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:55.527484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4680df9-fc64-44c9-b4a1-d6688f581e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:55.528101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4680df9-fc64-44c9-b4a1-d6688f581e82 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:55.528378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4680df9-fc64-44c9-b4a1-d6688f581e82 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:55.528672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4680df9-fc64-44c9-b4a1-d6688f581e82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:55.540616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 78/167] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:55.543716 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4c5c6-1ccb-4b91-8047-390b40aa6a83 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:55.604554 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:55.709913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4680df9-fc64-44c9-b4a1-d6688f581e82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:55.711465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4680df9-fc64-44c9-b4a1-d6688f581e82 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:55.732512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4c5c6-1ccb-4b91-8047-390b40aa6a83 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:55.818570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d5f62-6443-4056-8481-149126df4088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:55.819608 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d5f62-6443-4056-8481-149126df4088 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:55.820094 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d5f62-6443-4056-8481-149126df4088 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:55.820566 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d5f62-6443-4056-8481-149126df4088 None admin] PATH_INFO: `/v3/users/54d0a424442447c49a385a116cc9eec2` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:55.821976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d5f62-6443-4056-8481-149126df4088 None admin] RBAC: Authorizing `identity:get_user(user_id=54d0a424442447c49a385a116cc9eec2)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:55.824085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d5f62-6443-4056-8481-149126df4088 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:55.826708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 79/168] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:55 2022] GET /identity/v3/users/54d0a424442447c49a385a116cc9eec2 => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:55.833040 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d5f62-6443-4056-8481-149126df4088 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:55.947212 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 90/169] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:31:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:31:55.953795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4680df9-fc64-44c9-b4a1-d6688f581e82 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:55.998105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d5f62-6443-4056-8481-149126df4088 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:56.008710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:56.025320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:56.062248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd203089-dbf9-4f61-b1c7-175a88f0feb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:56.062926 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd203089-dbf9-4f61-b1c7-175a88f0feb1 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:56.063195 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd203089-dbf9-4f61-b1c7-175a88f0feb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:56.063474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd203089-dbf9-4f61-b1c7-175a88f0feb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:56.169754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4680df9-fc64-44c9-b4a1-d6688f581e82 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:56.208113 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd203089-dbf9-4f61-b1c7-175a88f0feb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:56.209833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd203089-dbf9-4f61-b1c7-175a88f0feb1 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:56.248511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77988304-a8f9-4fb1-a0d0-a6d927cc1d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:56.249199 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77988304-a8f9-4fb1-a0d0-a6d927cc1d1b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:56.249477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77988304-a8f9-4fb1-a0d0-a6d927cc1d1b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:56.249745 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77988304-a8f9-4fb1-a0d0-a6d927cc1d1b None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:56.250907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77988304-a8f9-4fb1-a0d0-a6d927cc1d1b None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:56.252356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77988304-a8f9-4fb1-a0d0-a6d927cc1d1b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:56.254194 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 91/170] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:55 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:31:56.254772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77988304-a8f9-4fb1-a0d0-a6d927cc1d1b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:56.396924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 80/171] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:31:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:31:56.398069 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd203089-dbf9-4f61-b1c7-175a88f0feb1 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:56.411165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:56.478747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:56.482027 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77988304-a8f9-4fb1-a0d0-a6d927cc1d1b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:56.546971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25c0f4-c65a-47af-9767-444ce72c788f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:56.547610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25c0f4-c65a-47af-9767-444ce72c788f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:56.547887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25c0f4-c65a-47af-9767-444ce72c788f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:56.548157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25c0f4-c65a-47af-9767-444ce72c788f None admin] PATH_INFO: `/v3/roles/member` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:56.560096 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25c0f4-c65a-47af-9767-444ce72c788f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-cd25c0f4-c65a-47af-9767-444ce72c788f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:56.566165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:56.567049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:56.567830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:56.567830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:56.567830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:56.567830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:56.567830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:56.567830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:56.567830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:56.567830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:56.567830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:56.567830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:56.567830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:31:56.567830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:31:56.567830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 92/172] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:56.569262 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25c0f4-c65a-47af-9767-444ce72c788f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:56.670758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd203089-dbf9-4f61-b1c7-175a88f0feb1 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:56.682464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25c0f4-c65a-47af-9767-444ce72c788f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:56.721071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae6e12-a081-4313-b44a-02673ec68ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:56.722298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae6e12-a081-4313-b44a-02673ec68ad8 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:56.722583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae6e12-a081-4313-b44a-02673ec68ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:56.722861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae6e12-a081-4313-b44a-02673ec68ad8 None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:56.724360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae6e12-a081-4313-b44a-02673ec68ad8 None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:56.726449 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae6e12-a081-4313-b44a-02673ec68ad8 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:56.728338 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 93/173] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:56 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:56.729549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92f408-e2ff-4519-831f-3f57e8d11d50 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:56.729832 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92f408-e2ff-4519-831f-3f57e8d11d50 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:56.730110 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92f408-e2ff-4519-831f-3f57e8d11d50 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:56.731695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 94/174] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:56.732316 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92f408-e2ff-4519-831f-3f57e8d11d50 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:56.769898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79f484-5c51-4a5a-9ed8-cac779cd12a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:56.770107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:56.770526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79f484-5c51-4a5a-9ed8-cac779cd12a1 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:56.770811 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79f484-5c51-4a5a-9ed8-cac779cd12a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:56.771653 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79f484-5c51-4a5a-9ed8-cac779cd12a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:56.772558 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79f484-5c51-4a5a-9ed8-cac779cd12a1 None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74, user.id=7275460c8c2e486f8768d1910d8aaa60) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:56.772902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79f484-5c51-4a5a-9ed8-cac779cd12a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:56.774347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79f484-5c51-4a5a-9ed8-cac779cd12a1 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:56.790847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 81/175] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:31:56 2022] GET /identity/v3/role_assignments?user.id=7275460c8c2e486f8768d1910d8aaa60&scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 608 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:56.791787 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79f484-5c51-4a5a-9ed8-cac779cd12a1 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:56.864096 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92f408-e2ff-4519-831f-3f57e8d11d50 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:56.941344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fe977-3f86-4e50-8ec1-603c843098b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:56.941983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fe977-3f86-4e50-8ec1-603c843098b1 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:56.942264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fe977-3f86-4e50-8ec1-603c843098b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:56.942539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fe977-3f86-4e50-8ec1-603c843098b1 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:56.943767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9fe977-3f86-4e50-8ec1-603c843098b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:56.944073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9fe977-3f86-4e50-8ec1-603c843098b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:56.948162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9fe977-3f86-4e50-8ec1-603c843098b1 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:56.967658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:56.969866 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 95/176] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:31:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:56.982206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2775ea2-6f00-40bb-9001-80c930700b63 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:56.982206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2775ea2-6f00-40bb-9001-80c930700b63 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:56.982206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2775ea2-6f00-40bb-9001-80c930700b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:57.009630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-f2775ea2-6f00-40bb-9001-80c930700b63 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:57.010846 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f2775ea2-6f00-40bb-9001-80c930700b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:57.029765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79f484-5c51-4a5a-9ed8-cac779cd12a1 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:57.068156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b284cf-6eab-44fe-8b9b-3bf704061977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:57.069982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b284cf-6eab-44fe-8b9b-3bf704061977 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:57.070569 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b284cf-6eab-44fe-8b9b-3bf704061977 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:57.071040 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b284cf-6eab-44fe-8b9b-3bf704061977 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:57.071961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b284cf-6eab-44fe-8b9b-3bf704061977 None admin] RBAC: Adding query filter params (role.id=8183d3986255476f959797314695f92a, scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74, user.id=54d0a424442447c49a385a116cc9eec2) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:57.072455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b284cf-6eab-44fe-8b9b-3bf704061977 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:57.074260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b284cf-6eab-44fe-8b9b-3bf704061977 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:57.083246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 82/177] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:31:56 2022] GET /identity/v3/role_assignments?user.id=54d0a424442447c49a385a116cc9eec2&scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74&role.id=8183d3986255476f959797314695f92a => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:57.084765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b284cf-6eab-44fe-8b9b-3bf704061977 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:57.106736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 96/178] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:57.110955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e69d69-ec49-4b8a-820e-ae36649ef7de None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:57.110955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e69d69-ec49-4b8a-820e-ae36649ef7de None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:57.110955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e69d69-ec49-4b8a-820e-ae36649ef7de None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:57.110955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 97/179] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:57.111821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c8489-d477-4749-9290-cf1f45cb123c None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:57.112109 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c8489-d477-4749-9290-cf1f45cb123c None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:57.112390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c8489-d477-4749-9290-cf1f45cb123c None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:57.117665 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 98/180] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:57.119651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac945-35d6-4a44-8c6b-1713ad458100 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:57.119651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac945-35d6-4a44-8c6b-1713ad458100 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:57.119651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac945-35d6-4a44-8c6b-1713ad458100 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:57.122235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 99/181] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:31:57.127689 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbafc47-7178-4eb0-baed-fc65025c0618 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:57.127689 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbafc47-7178-4eb0-baed-fc65025c0618 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:57.127689 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbafc47-7178-4eb0-baed-fc65025c0618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:57.144171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0cbafc47-7178-4eb0-baed-fc65025c0618 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:57.145429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0cbafc47-7178-4eb0-baed-fc65025c0618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:57.164438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:57.234005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 100/182] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:57.235581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31008628-586d-45be-a6f4-5457009b9e19 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:57.235908 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b284cf-6eab-44fe-8b9b-3bf704061977 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:57.236206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31008628-586d-45be-a6f4-5457009b9e19 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:57.236629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31008628-586d-45be-a6f4-5457009b9e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:57.259831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-31008628-586d-45be-a6f4-5457009b9e19 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:57.262066 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-31008628-586d-45be-a6f4-5457009b9e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:57.276852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c34ca8-d6ac-4221-8dc3-74227930f365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:57.277588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c34ca8-d6ac-4221-8dc3-74227930f365 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:57.277860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c34ca8-d6ac-4221-8dc3-74227930f365 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:57.278195 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c34ca8-d6ac-4221-8dc3-74227930f365 None admin] PATH_INFO: `/v3/users/54d0a424442447c49a385a116cc9eec2` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:57.279284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c34ca8-d6ac-4221-8dc3-74227930f365 None admin] RBAC: Authorizing `identity:get_user(user_id=54d0a424442447c49a385a116cc9eec2)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:57.280800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c34ca8-d6ac-4221-8dc3-74227930f365 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:57.287775 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 83/183] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:57 2022] GET /identity/v3/users/54d0a424442447c49a385a116cc9eec2 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:57.288867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a0856-33b9-48fe-b989-6421c1f757e9 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:57.289191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a0856-33b9-48fe-b989-6421c1f757e9 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:57.289434 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a0856-33b9-48fe-b989-6421c1f757e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:57.304954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:57.311461 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-635a0856-33b9-48fe-b989-6421c1f757e9 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:31:57.313539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-635a0856-33b9-48fe-b989-6421c1f757e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:31:57.343985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 101/184] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:57.344692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31008628-586d-45be-a6f4-5457009b9e19 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:57.433161 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 84/185] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:31:57.447705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:57.452413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a0856-33b9-48fe-b989-6421c1f757e9 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:57.478098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31008628-586d-45be-a6f4-5457009b9e19 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:57.522632 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c043e3-97f8-4ff3-8af1-e8e11ffb56fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:57.523079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c043e3-97f8-4ff3-8af1-e8e11ffb56fd None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:57.523361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c043e3-97f8-4ff3-8af1-e8e11ffb56fd None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:57.523636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c043e3-97f8-4ff3-8af1-e8e11ffb56fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:57.640078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c043e3-97f8-4ff3-8af1-e8e11ffb56fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:57.640459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a0856-33b9-48fe-b989-6421c1f757e9 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:57.641592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c043e3-97f8-4ff3-8af1-e8e11ffb56fd None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:57.678174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:57.684267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021f826-08f6-43bd-8f00-1336a0c62c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:57.684950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021f826-08f6-43bd-8f00-1336a0c62c68 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:57.685326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021f826-08f6-43bd-8f00-1336a0c62c68 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:57.685601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021f826-08f6-43bd-8f00-1336a0c62c68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:57.696260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021f826-08f6-43bd-8f00-1336a0c62c68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-c021f826-08f6-43bd-8f00-1336a0c62c68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:57.701179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:57.702303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:57.703355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:57.703355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:57.703355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:57.703355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:57.703355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:57.703355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:57.703355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:57.703355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:57.703355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:57.703355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:57.703355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:31:57.703355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:31:57.704121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 85/186] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:31:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:31:57.704895 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021f826-08f6-43bd-8f00-1336a0c62c68 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:57.736445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:57.760842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 102/187] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:31:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:31:57.760842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c043e3-97f8-4ff3-8af1-e8e11ffb56fd None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:57.817397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021f826-08f6-43bd-8f00-1336a0c62c68 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:57.851041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e504f2-2e33-4116-85cf-261d54653751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:57.851691 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e504f2-2e33-4116-85cf-261d54653751 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:57.851973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e504f2-2e33-4116-85cf-261d54653751 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:57.852244 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e504f2-2e33-4116-85cf-261d54653751 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:57.856561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c043e3-97f8-4ff3-8af1-e8e11ffb56fd None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:57.862286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e504f2-2e33-4116-85cf-261d54653751 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-66e504f2-2e33-4116-85cf-261d54653751 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:57.867565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:57.868629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:57.873913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:57.873913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:57.873913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:57.873913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:57.873913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:57.873913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:57.873913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:57.873913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:57.873913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:57.873913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:57.873913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:31:57.873913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:31:57.873913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 86/188] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:31:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:57.873913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e504f2-2e33-4116-85cf-261d54653751 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:57.891674 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d7f5e-b20d-449e-ae5c-458cfb0adc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:57.892294 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d7f5e-b20d-449e-ae5c-458cfb0adc29 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:57.892571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d7f5e-b20d-449e-ae5c-458cfb0adc29 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:57.892868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d7f5e-b20d-449e-ae5c-458cfb0adc29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:57.893958 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d7f5e-b20d-449e-ae5c-458cfb0adc29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:57.895381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d7f5e-b20d-449e-ae5c-458cfb0adc29 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:57.902381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 103/189] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:31:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:57.902381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d7f5e-b20d-449e-ae5c-458cfb0adc29 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:57.925669 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:57.948376 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:57.972072 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e504f2-2e33-4116-85cf-261d54653751 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:57.998726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d7f5e-b20d-449e-ae5c-458cfb0adc29 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:58.006284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b49b9-bf9f-4a11-a42e-dc3ea619ba6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:58.006986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b49b9-bf9f-4a11-a42e-dc3ea619ba6a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:58.007264 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b49b9-bf9f-4a11-a42e-dc3ea619ba6a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:58.007539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b49b9-bf9f-4a11-a42e-dc3ea619ba6a None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:58.008198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b49b9-bf9f-4a11-a42e-dc3ea619ba6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:58.008502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b49b9-bf9f-4a11-a42e-dc3ea619ba6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:58.009972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b49b9-bf9f-4a11-a42e-dc3ea619ba6a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:58.017967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 87/190] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:58.018841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b49b9-bf9f-4a11-a42e-dc3ea619ba6a None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:58.036361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d27dd-2dfc-444b-8510-0d773cf53fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:58.037713 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d27dd-2dfc-444b-8510-0d773cf53fb3 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:58.037997 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d27dd-2dfc-444b-8510-0d773cf53fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:58.038272 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d27dd-2dfc-444b-8510-0d773cf53fb3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:58.048262 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d27dd-2dfc-444b-8510-0d773cf53fb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-9e2d27dd-2dfc-444b-8510-0d773cf53fb3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:31:58.053092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:31:58.054017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:31:58.055022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:31:58.055022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:31:58.055022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:31:58.055022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:31:58.055022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:31:58.055022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:31:58.055022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:31:58.055022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:31:58.055022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:31:58.055022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:31:58.055022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:31:58.055022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:31:58.055022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 104/191] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:31:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:31:58.055022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d27dd-2dfc-444b-8510-0d773cf53fb3 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:58.120534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b49b9-bf9f-4a11-a42e-dc3ea619ba6a None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:58.134776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:58.153695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d27dd-2dfc-444b-8510-0d773cf53fb3 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:58.154184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf5aa5-bc26-45d2-9837-f6a12bd695a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:58.154812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf5aa5-bc26-45d2-9837-f6a12bd695a1 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:58.155097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf5aa5-bc26-45d2-9837-f6a12bd695a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:58.155372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf5aa5-bc26-45d2-9837-f6a12bd695a1 None admin] PATH_INFO: `/v3/projects/772bf126b9df4caaac87f3a14bfd2d32` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:58.156501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf5aa5-bc26-45d2-9837-f6a12bd695a1 None admin] RBAC: Authorizing `identity:get_project(project_id=772bf126b9df4caaac87f3a14bfd2d32)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:58.158031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf5aa5-bc26-45d2-9837-f6a12bd695a1 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:58.159664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 88/192] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:58 2022] GET /identity/v3/projects/772bf126b9df4caaac87f3a14bfd2d32 => generated 312 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:58.160406 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf5aa5-bc26-45d2-9837-f6a12bd695a1 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:58.165154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:58.188577 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e53b3-de6b-4c04-ab74-c05166629818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:58.189211 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e53b3-de6b-4c04-ab74-c05166629818 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:58.189497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e53b3-de6b-4c04-ab74-c05166629818 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:58.189770 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e53b3-de6b-4c04-ab74-c05166629818 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:58.190432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e53b3-de6b-4c04-ab74-c05166629818 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:58.190734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e53b3-de6b-4c04-ab74-c05166629818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:58.192121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e53b3-de6b-4c04-ab74-c05166629818 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:58.201111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 105/193] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:31:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:58.201905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e53b3-de6b-4c04-ab74-c05166629818 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:58.262821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf5aa5-bc26-45d2-9837-f6a12bd695a1 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:58.295181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb07e03-b4e8-44e5-80ee-b695ca323c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:58.295791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb07e03-b4e8-44e5-80ee-b695ca323c0d None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:58.296076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb07e03-b4e8-44e5-80ee-b695ca323c0d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:58.296348 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb07e03-b4e8-44e5-80ee-b695ca323c0d None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:58.297025 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb07e03-b4e8-44e5-80ee-b695ca323c0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:58.298475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb07e03-b4e8-44e5-80ee-b695ca323c0d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:58.302064 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e53b3-de6b-4c04-ab74-c05166629818 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:58.334986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba81fda-48c9-4bdb-b606-c4ecfe396b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:58.335600 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba81fda-48c9-4bdb-b606-c4ecfe396b77 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:58.335876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba81fda-48c9-4bdb-b606-c4ecfe396b77 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:58.336144 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba81fda-48c9-4bdb-b606-c4ecfe396b77 None admin] PATH_INFO: `/v3/users/8bbbf977dd7347f7a7527745e4da29c5` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:58.348931 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba81fda-48c9-4bdb-b606-c4ecfe396b77 None admin] RBAC: Authorizing `identity:get_user(user_id=8bbbf977dd7347f7a7527745e4da29c5)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:58.350513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba81fda-48c9-4bdb-b606-c4ecfe396b77 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:58.352108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 106/194] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:58 2022] GET /identity/v3/users/8bbbf977dd7347f7a7527745e4da29c5 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:58.352993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba81fda-48c9-4bdb-b606-c4ecfe396b77 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:58.379845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:58.388159 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 89/195] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:31:58 2022] POST /identity/v3/users => generated 297 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:31:58.389023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb07e03-b4e8-44e5-80ee-b695ca323c0d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:58.447603 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:58.462593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba81fda-48c9-4bdb-b606-c4ecfe396b77 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:58.501294 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f966c13-78d6-46ec-9360-6e29f611511f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:58.501910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f966c13-78d6-46ec-9360-6e29f611511f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:58.502196 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f966c13-78d6-46ec-9360-6e29f611511f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:58.502474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f966c13-78d6-46ec-9360-6e29f611511f None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:58.503216 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f966c13-78d6-46ec-9360-6e29f611511f None admin] RBAC: Adding query filter params (name=reader) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:58.503481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f966c13-78d6-46ec-9360-6e29f611511f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:58.504900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f966c13-78d6-46ec-9360-6e29f611511f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:58.516702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 107/196] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:31:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:58.516702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f966c13-78d6-46ec-9360-6e29f611511f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:58.522030 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb07e03-b4e8-44e5-80ee-b695ca323c0d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:58.563767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52f497-1a69-4fa1-bb21-713abc81c1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:58.564376 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52f497-1a69-4fa1-bb21-713abc81c1a4 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:58.568477 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52f497-1a69-4fa1-bb21-713abc81c1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:58.568477 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52f497-1a69-4fa1-bb21-713abc81c1a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:58.568477 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52f497-1a69-4fa1-bb21-713abc81c1a4 None admin] RBAC: Adding query filter params (role.id=9fe465cfe8764112a4cc92f3480326b8, scope.project.id=772bf126b9df4caaac87f3a14bfd2d32, user.id=54d0a424442447c49a385a116cc9eec2) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:58.568477 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52f497-1a69-4fa1-bb21-713abc81c1a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:58.568477 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52f497-1a69-4fa1-bb21-713abc81c1a4 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:58.575805 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 90/197] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:31:58 2022] GET /identity/v3/role_assignments?user.id=54d0a424442447c49a385a116cc9eec2&scope.project.id=772bf126b9df4caaac87f3a14bfd2d32&role.id=9fe465cfe8764112a4cc92f3480326b8 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:58.581920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52f497-1a69-4fa1-bb21-713abc81c1a4 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:58.650333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:58.662549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f966c13-78d6-46ec-9360-6e29f611511f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:58.705313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ad4d0-89b2-4aab-a995-2e970a10e094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:58.706062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ad4d0-89b2-4aab-a995-2e970a10e094 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:58.706894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ad4d0-89b2-4aab-a995-2e970a10e094 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:58.706894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ad4d0-89b2-4aab-a995-2e970a10e094 None admin] PATH_INFO: `/v3/users/8bbbf977dd7347f7a7527745e4da29c5` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:58.710665 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ad4d0-89b2-4aab-a995-2e970a10e094 None admin] RBAC: Authorizing `identity:get_user(user_id=8bbbf977dd7347f7a7527745e4da29c5)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:58.712703 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ad4d0-89b2-4aab-a995-2e970a10e094 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:58.714573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 108/198] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:58 2022] GET /identity/v3/users/8bbbf977dd7347f7a7527745e4da29c5 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:58.715496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ad4d0-89b2-4aab-a995-2e970a10e094 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:58.731912 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52f497-1a69-4fa1-bb21-713abc81c1a4 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:58.733927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:58.793395 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb1249-daaa-4beb-af33-cc73ce9212c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:58.794030 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1249-daaa-4beb-af33-cc73ce9212c5 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:58.794320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1249-daaa-4beb-af33-cc73ce9212c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:58.795173 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1249-daaa-4beb-af33-cc73ce9212c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:58.857626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ad4d0-89b2-4aab-a995-2e970a10e094 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:58.895018 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb1249-daaa-4beb-af33-cc73ce9212c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:58.897304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb1249-daaa-4beb-af33-cc73ce9212c5 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:58.904723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112de0c3-4def-4989-ac1b-c778934b0575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:58.905374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112de0c3-4def-4989-ac1b-c778934b0575 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:58.905656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112de0c3-4def-4989-ac1b-c778934b0575 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:58.905941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112de0c3-4def-4989-ac1b-c778934b0575 None admin] PATH_INFO: `/v3/users/d3115fa7537242488412fff3edfdebce` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:58.908072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112de0c3-4def-4989-ac1b-c778934b0575 None admin] RBAC: Authorizing `identity:get_user(user_id=d3115fa7537242488412fff3edfdebce)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:58.911180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112de0c3-4def-4989-ac1b-c778934b0575 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:58.914582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 109/199] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:31:58 2022] GET /identity/v3/users/d3115fa7537242488412fff3edfdebce => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:58.915554 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112de0c3-4def-4989-ac1b-c778934b0575 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:58.957054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:59.034363 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 91/200] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:31:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:31:59.035231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb1249-daaa-4beb-af33-cc73ce9212c5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:59.041212 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:59.058624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112de0c3-4def-4989-ac1b-c778934b0575 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:59.097553 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e8e96-deee-4334-bce5-f1e138c37e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:59.098667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e8e96-deee-4334-bce5-f1e138c37e9b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:59.098971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e8e96-deee-4334-bce5-f1e138c37e9b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:59.099307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e8e96-deee-4334-bce5-f1e138c37e9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:59.155350 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb1249-daaa-4beb-af33-cc73ce9212c5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:59.194599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0b1ab-b243-47ba-9b57-c35c08945e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:59.195214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0b1ab-b243-47ba-9b57-c35c08945e33 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:59.195495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0b1ab-b243-47ba-9b57-c35c08945e33 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:59.195753 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e8e96-deee-4334-bce5-f1e138c37e9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:59.195929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0b1ab-b243-47ba-9b57-c35c08945e33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:59.197258 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e8e96-deee-4334-bce5-f1e138c37e9b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:59.233379 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:59.282598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d0b1ab-b243-47ba-9b57-c35c08945e33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:59.284099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d0b1ab-b243-47ba-9b57-c35c08945e33 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:59.286202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:59.329302 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 110/201] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:31:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:31:59.330151 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e8e96-deee-4334-bce5-f1e138c37e9b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:59.405004 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 92/202] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:31:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:31:59.405964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0b1ab-b243-47ba-9b57-c35c08945e33 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:59.442485 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e8e96-deee-4334-bce5-f1e138c37e9b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:59.468076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:59.477809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2388639-c3a9-4a37-b20a-771c6d203ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:59.478438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2388639-c3a9-4a37-b20a-771c6d203ec0 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:59.478716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2388639-c3a9-4a37-b20a-771c6d203ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:59.478990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2388639-c3a9-4a37-b20a-771c6d203ec0 None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:59.481303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2388639-c3a9-4a37-b20a-771c6d203ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:59.481845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2388639-c3a9-4a37-b20a-771c6d203ec0 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:59.483835 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 111/203] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:59 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:59.484669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2388639-c3a9-4a37-b20a-771c6d203ec0 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:59.518594 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0b1ab-b243-47ba-9b57-c35c08945e33 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:59.520269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:59.561245 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc5277-8d3f-4ea7-a7bb-db6a69c80383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:59.561973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc5277-8d3f-4ea7-a7bb-db6a69c80383 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:59.562421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc5277-8d3f-4ea7-a7bb-db6a69c80383 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:59.562939 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc5277-8d3f-4ea7-a7bb-db6a69c80383 None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:59.564294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc5277-8d3f-4ea7-a7bb-db6a69c80383 None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:59.566059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc5277-8d3f-4ea7-a7bb-db6a69c80383 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:59.567973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 93/204] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:59 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:59.569070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc5277-8d3f-4ea7-a7bb-db6a69c80383 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:59.605073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2388639-c3a9-4a37-b20a-771c6d203ec0 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:59.640507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc979b-48d5-47fd-956c-8cc2d6193315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:59.641174 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc979b-48d5-47fd-956c-8cc2d6193315 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:59.641460 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc979b-48d5-47fd-956c-8cc2d6193315 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:59.641732 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc979b-48d5-47fd-956c-8cc2d6193315 None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:59.642878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc979b-48d5-47fd-956c-8cc2d6193315 None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:59.644310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc979b-48d5-47fd-956c-8cc2d6193315 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:59.646512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 112/205] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:31:59 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:59.647691 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc979b-48d5-47fd-956c-8cc2d6193315 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:59.683193 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc5277-8d3f-4ea7-a7bb-db6a69c80383 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:59.712401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:59.722671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23399def-486b-488d-9684-6f6097271825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:59.727445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23399def-486b-488d-9684-6f6097271825 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:59.727445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23399def-486b-488d-9684-6f6097271825 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:59.727445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23399def-486b-488d-9684-6f6097271825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:59.727445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23399def-486b-488d-9684-6f6097271825 None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=7ca3316be8414d20996d0efd2dceee60, user.id=8bbbf977dd7347f7a7527745e4da29c5) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:59.727445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23399def-486b-488d-9684-6f6097271825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:59.728727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23399def-486b-488d-9684-6f6097271825 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:59.735343 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 94/206] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:31:59 2022] GET /identity/v3/role_assignments?user.id=8bbbf977dd7347f7a7527745e4da29c5&scope.project.id=7ca3316be8414d20996d0efd2dceee60&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:59.736544 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23399def-486b-488d-9684-6f6097271825 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:31:59.775887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc979b-48d5-47fd-956c-8cc2d6193315 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:59.780004 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:31:59.866363 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23399def-486b-488d-9684-6f6097271825 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:31:59.874166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2f140-4721-4f87-8bf8-0a5d4f7f84cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:59.874801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2f140-4721-4f87-8bf8-0a5d4f7f84cf None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:59.875085 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2f140-4721-4f87-8bf8-0a5d4f7f84cf None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:59.875359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2f140-4721-4f87-8bf8-0a5d4f7f84cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:59.876451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2f140-4721-4f87-8bf8-0a5d4f7f84cf None admin] RBAC: Adding query filter params (role.id=8183d3986255476f959797314695f92a, scope.project.id=7ca3316be8414d20996d0efd2dceee60, user.id=8bbbf977dd7347f7a7527745e4da29c5) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:59.876451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2f140-4721-4f87-8bf8-0a5d4f7f84cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:59.877893 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2f140-4721-4f87-8bf8-0a5d4f7f84cf None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:59.898743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 113/207] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:31:59 2022] GET /identity/v3/role_assignments?user.id=8bbbf977dd7347f7a7527745e4da29c5&scope.project.id=7ca3316be8414d20996d0efd2dceee60&role.id=8183d3986255476f959797314695f92a => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:31:59.909297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a4f1b-e95d-4852-81b2-9f40efa41af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:31:59.910455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a4f1b-e95d-4852-81b2-9f40efa41af7 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:31:59.910938 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a4f1b-e95d-4852-81b2-9f40efa41af7 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:31:59.911409 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a4f1b-e95d-4852-81b2-9f40efa41af7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:31:59.912428 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a4f1b-e95d-4852-81b2-9f40efa41af7 None admin] RBAC: Adding query filter params (role.id=27470a2fbc754a179323bb892496b013, scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74, user.id=d3115fa7537242488412fff3edfdebce) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:31:59.913008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a4f1b-e95d-4852-81b2-9f40efa41af7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:31:59.914687 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a4f1b-e95d-4852-81b2-9f40efa41af7 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:31:59.922104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 95/208] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:31:59 2022] GET /identity/v3/role_assignments?user.id=d3115fa7537242488412fff3edfdebce&scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74&role.id=27470a2fbc754a179323bb892496b013 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:00.623442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49679e84-78ba-4164-8980-0691bd1c0174 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:00.624438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49679e84-78ba-4164-8980-0691bd1c0174 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:00.624960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49679e84-78ba-4164-8980-0691bd1c0174 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:00.626509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 114/209] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:00.631952 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fff913-5681-4df6-867e-9e0408cc0610 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:00.632792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fff913-5681-4df6-867e-9e0408cc0610 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:00.633294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fff913-5681-4df6-867e-9e0408cc0610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:00.652148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-56fff913-5681-4df6-867e-9e0408cc0610 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:00.653677 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-56fff913-5681-4df6-867e-9e0408cc0610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:00.695164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6a99-d928-4122-bc54-156da49f4232 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:00.695462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6a99-d928-4122-bc54-156da49f4232 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:00.695718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6a99-d928-4122-bc54-156da49f4232 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:00.697208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 115/210] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:32:00.702101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02e73-c259-4aba-ab13-361b7a593cb1 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:00.702318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02e73-c259-4aba-ab13-361b7a593cb1 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:00.702596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02e73-c259-4aba-ab13-361b7a593cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:00.721182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-08b02e73-c259-4aba-ab13-361b7a593cb1 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:00.722395 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-08b02e73-c259-4aba-ab13-361b7a593cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:00.730327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 96/211] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:00.792633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 116/212] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:32:01.002106 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fff913-5681-4df6-867e-9e0408cc0610 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:01.034897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:01.066647 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b02e73-c259-4aba-ab13-361b7a593cb1 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:01.113349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fff913-5681-4df6-867e-9e0408cc0610 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:01.128352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:01.150831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f27fc-28a1-4304-935b-ff9d89e947b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:01.151781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f27fc-28a1-4304-935b-ff9d89e947b5 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:01.152253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f27fc-28a1-4304-935b-ff9d89e947b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:01.152770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f27fc-28a1-4304-935b-ff9d89e947b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:01.164032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f27fc-28a1-4304-935b-ff9d89e947b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-593f27fc-28a1-4304-935b-ff9d89e947b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:01.168960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:01.169815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:01.170587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:01.170587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:01.170587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:01.170587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:01.170587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:01.170587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:01.170587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:01.170587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:01.170587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:01.170587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:01.170587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:01.170587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:01.173067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 97/213] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:01.178562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b02e73-c259-4aba-ab13-361b7a593cb1 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:01.179604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f27fc-28a1-4304-935b-ff9d89e947b5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:01.214492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41016c82-001d-4956-8e75-b5611c5668bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:01.215245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41016c82-001d-4956-8e75-b5611c5668bb None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:01.215514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41016c82-001d-4956-8e75-b5611c5668bb None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:01.215791 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41016c82-001d-4956-8e75-b5611c5668bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:01.217303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41016c82-001d-4956-8e75-b5611c5668bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:01.218495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41016c82-001d-4956-8e75-b5611c5668bb None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:01.225192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 117/214] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:01.226086 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41016c82-001d-4956-8e75-b5611c5668bb None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:01.275547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:01.288396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f27fc-28a1-4304-935b-ff9d89e947b5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:01.326831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e081a-21e2-457c-8305-96c3eae5c373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:01.327795 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e081a-21e2-457c-8305-96c3eae5c373 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:01.328267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e081a-21e2-457c-8305-96c3eae5c373 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:01.328794 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e081a-21e2-457c-8305-96c3eae5c373 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:01.329641 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e081a-21e2-457c-8305-96c3eae5c373 None admin] RBAC: Adding query filter params (name=member) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:01.330134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e081a-21e2-457c-8305-96c3eae5c373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:01.331702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e081a-21e2-457c-8305-96c3eae5c373 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:01.335469 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41016c82-001d-4956-8e75-b5611c5668bb None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:01.344681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 98/215] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:01.347043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e081a-21e2-457c-8305-96c3eae5c373 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:01.361243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:01.373844 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ae324-e68a-4078-a5bd-1011bf0f40ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:01.374453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ae324-e68a-4078-a5bd-1011bf0f40ec None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:01.377900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ae324-e68a-4078-a5bd-1011bf0f40ec None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:01.377900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ae324-e68a-4078-a5bd-1011bf0f40ec None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:01.377900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ae324-e68a-4078-a5bd-1011bf0f40ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:01.377900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ae324-e68a-4078-a5bd-1011bf0f40ec None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:01.458344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e081a-21e2-457c-8305-96c3eae5c373 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:01.466133 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 118/216] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:32:01 2022] POST /identity/v3/users => generated 297 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:32:01.508967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c0df7-b6d2-4b05-909a-ec634e770792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:01.510022 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c0df7-b6d2-4b05-909a-ec634e770792 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:01.510516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c0df7-b6d2-4b05-909a-ec634e770792 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:01.511019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c0df7-b6d2-4b05-909a-ec634e770792 None admin] PATH_INFO: `/v3/users/b30c7335275449749c54c6485828c374` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:01.533866 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c0df7-b6d2-4b05-909a-ec634e770792 None admin] RBAC: Authorizing `identity:get_user(user_id=b30c7335275449749c54c6485828c374)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:01.537345 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c0df7-b6d2-4b05-909a-ec634e770792 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:01.539915 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 99/217] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:01 2022] GET /identity/v3/users/b30c7335275449749c54c6485828c374 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:01.545849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ae324-e68a-4078-a5bd-1011bf0f40ec None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:01.663145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ae324-e68a-4078-a5bd-1011bf0f40ec None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:01.698018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009582f-6bb1-4891-afe5-53afa80fea53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:01.698636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009582f-6bb1-4891-afe5-53afa80fea53 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:01.698916 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009582f-6bb1-4891-afe5-53afa80fea53 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:01.699188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009582f-6bb1-4891-afe5-53afa80fea53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:01.751988 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:01.774932 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009582f-6bb1-4891-afe5-53afa80fea53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:01.776743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009582f-6bb1-4891-afe5-53afa80fea53 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:01.886430 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 119/218] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:01.892142 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c0df7-b6d2-4b05-909a-ec634e770792 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:01.895566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14994ff-a13d-48fb-9fe7-e5731e4533cd None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:01.896422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14994ff-a13d-48fb-9fe7-e5731e4533cd None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:01.896755 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14994ff-a13d-48fb-9fe7-e5731e4533cd None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:01.901614 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 120/219] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:01.907887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531536be-7c23-4177-8141-1d0acfc675f8 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:01.909442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531536be-7c23-4177-8141-1d0acfc675f8 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:01.909727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531536be-7c23-4177-8141-1d0acfc675f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:01.917383 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:01.930275 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-531536be-7c23-4177-8141-1d0acfc675f8 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:01.931481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-531536be-7c23-4177-8141-1d0acfc675f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:02.003946 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 121/220] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:02.005467 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877a581-3d45-49dc-8f13-c26adcff3561 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:02.005768 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877a581-3d45-49dc-8f13-c26adcff3561 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:02.006045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877a581-3d45-49dc-8f13-c26adcff3561 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:02.007644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 122/221] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:02.016715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c0df7-b6d2-4b05-909a-ec634e770792 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:02.020472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab789e-56d5-4f01-b127-dfd56de81aca None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:02.021562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab789e-56d5-4f01-b127-dfd56de81aca None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:02.022019 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab789e-56d5-4f01-b127-dfd56de81aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:02.040580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-b7ab789e-56d5-4f01-b127-dfd56de81aca None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:02.042097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b7ab789e-56d5-4f01-b127-dfd56de81aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:02.046257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:02.054394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49347a16-bbff-42f4-a569-b83e71b1c2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:02.059771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49347a16-bbff-42f4-a569-b83e71b1c2ea None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:02.059771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49347a16-bbff-42f4-a569-b83e71b1c2ea None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:02.059771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49347a16-bbff-42f4-a569-b83e71b1c2ea None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:02.059771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49347a16-bbff-42f4-a569-b83e71b1c2ea None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:02.059771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49347a16-bbff-42f4-a569-b83e71b1c2ea None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:02.060153 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 100/222] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:01 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:02.065871 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49347a16-bbff-42f4-a569-b83e71b1c2ea None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:02.153106 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:02.175720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49347a16-bbff-42f4-a569-b83e71b1c2ea None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:02.215249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579f412-9a69-487c-a253-8030abafef31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:02.215959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579f412-9a69-487c-a253-8030abafef31 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:02.216306 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579f412-9a69-487c-a253-8030abafef31 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:02.216661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579f412-9a69-487c-a253-8030abafef31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:02.217665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579f412-9a69-487c-a253-8030abafef31 None admin] RBAC: Adding query filter params (role.id=9fe465cfe8764112a4cc92f3480326b8, scope.project.id=7ca3316be8414d20996d0efd2dceee60, user.id=b30c7335275449749c54c6485828c374) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:02.218091 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579f412-9a69-487c-a253-8030abafef31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:02.219907 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579f412-9a69-487c-a253-8030abafef31 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:02.226291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 123/223] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:32:02.233905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 101/224] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:32:02 2022] GET /identity/v3/role_assignments?user.id=b30c7335275449749c54c6485828c374&scope.project.id=7ca3316be8414d20996d0efd2dceee60&role.id=9fe465cfe8764112a4cc92f3480326b8 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:02.296105 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab789e-56d5-4f01-b127-dfd56de81aca None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:02.439949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab789e-56d5-4f01-b127-dfd56de81aca None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:02.476150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f86ce-1267-43c4-bdf1-f7536a0ed64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:02.477036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f86ce-1267-43c4-bdf1-f7536a0ed64a None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:02.477458 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f86ce-1267-43c4-bdf1-f7536a0ed64a None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:02.477892 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f86ce-1267-43c4-bdf1-f7536a0ed64a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:02.485827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:02.492940 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f86ce-1267-43c4-bdf1-f7536a0ed64a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-b37f86ce-1267-43c4-bdf1-f7536a0ed64a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:02.498048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:02.499107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:02.500101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:02.500101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:02.500101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:02.500101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:02.500101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:02.500101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:02.500101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:02.500101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:02.500101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:02.500101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:02.500101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:32:02.500101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:02.502418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 124/225] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:32:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:02.507808 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579f412-9a69-487c-a253-8030abafef31 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:02.618865 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579f412-9a69-487c-a253-8030abafef31 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:02.621482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f86ce-1267-43c4-bdf1-f7536a0ed64a None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:02.656020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dacfd-1f83-453b-9b12-fa2fc60e42ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:02.658914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dacfd-1f83-453b-9b12-fa2fc60e42ce None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:02.658914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dacfd-1f83-453b-9b12-fa2fc60e42ce None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:02.658914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dacfd-1f83-453b-9b12-fa2fc60e42ce None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:02.658914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488dacfd-1f83-453b-9b12-fa2fc60e42ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:02.658914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488dacfd-1f83-453b-9b12-fa2fc60e42ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:02.659855 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488dacfd-1f83-453b-9b12-fa2fc60e42ce None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:02.669104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 102/226] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:02.675114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dacfd-1f83-453b-9b12-fa2fc60e42ce None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:02.693144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:02.783301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dacfd-1f83-453b-9b12-fa2fc60e42ce None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:02.849284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956746a-0ca5-4af7-a744-0bcbb519a733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:02.851002 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956746a-0ca5-4af7-a744-0bcbb519a733 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:02.851286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956746a-0ca5-4af7-a744-0bcbb519a733 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:02.851566 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956746a-0ca5-4af7-a744-0bcbb519a733 None admin] PATH_INFO: `/v3/users/8bbbf977dd7347f7a7527745e4da29c5` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:02.853349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956746a-0ca5-4af7-a744-0bcbb519a733 None admin] RBAC: Authorizing `identity:get_user(user_id=8bbbf977dd7347f7a7527745e4da29c5)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:02.855134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956746a-0ca5-4af7-a744-0bcbb519a733 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:02.859016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 103/227] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:02 2022] GET /identity/v3/users/8bbbf977dd7347f7a7527745e4da29c5 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:02.863658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956746a-0ca5-4af7-a744-0bcbb519a733 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:02.868248 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f86ce-1267-43c4-bdf1-f7536a0ed64a None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:02.932398 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e5483-4bde-4ea6-bc08-dcc7e183c2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:02.933075 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e5483-4bde-4ea6-bc08-dcc7e183c2fa None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:02.933357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e5483-4bde-4ea6-bc08-dcc7e183c2fa None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:02.933628 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e5483-4bde-4ea6-bc08-dcc7e183c2fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:02.944195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e5483-4bde-4ea6-bc08-dcc7e183c2fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-9e0e5483-4bde-4ea6-bc08-dcc7e183c2fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:02.948843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:02.949707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:02.950495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:02.950495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:02.950495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:02.950495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:02.950495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:02.950495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:02.950495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:02.950495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:02.950495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:02.950495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:02.950495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:02.950495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:02.950495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 125/228] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:32:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:02.957572 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e5483-4bde-4ea6-bc08-dcc7e183c2fa None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:02.981669 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:03.048817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956746a-0ca5-4af7-a744-0bcbb519a733 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:03.084858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:03.123706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6acf5-ef54-4ada-99ba-c1007e877e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:03.124385 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6acf5-ef54-4ada-99ba-c1007e877e0a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:03.124689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6acf5-ef54-4ada-99ba-c1007e877e0a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:03.124967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6acf5-ef54-4ada-99ba-c1007e877e0a None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:03.127487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6acf5-ef54-4ada-99ba-c1007e877e0a None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:03.129775 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6acf5-ef54-4ada-99ba-c1007e877e0a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:03.132509 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 104/229] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:02 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:03.138868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6acf5-ef54-4ada-99ba-c1007e877e0a None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:03.174609 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e5483-4bde-4ea6-bc08-dcc7e183c2fa None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:03.231632 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d42ea-7d86-446c-92bc-2aabcba7a706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:03.232979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d42ea-7d86-446c-92bc-2aabcba7a706 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:03.233576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d42ea-7d86-446c-92bc-2aabcba7a706 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:03.236328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d42ea-7d86-446c-92bc-2aabcba7a706 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:03.237192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d42ea-7d86-446c-92bc-2aabcba7a706 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:03.237641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d42ea-7d86-446c-92bc-2aabcba7a706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:03.239172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d42ea-7d86-446c-92bc-2aabcba7a706 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:03.257175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 126/230] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:03.262779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d42ea-7d86-446c-92bc-2aabcba7a706 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:03.301895 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6acf5-ef54-4ada-99ba-c1007e877e0a None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:03.339985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cb4e2-9441-44bf-b7a9-7d097dc2bda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:03.340905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cb4e2-9441-44bf-b7a9-7d097dc2bda5 None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:03.341399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cb4e2-9441-44bf-b7a9-7d097dc2bda5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:03.341820 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cb4e2-9441-44bf-b7a9-7d097dc2bda5 None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60/users/8bbbf977dd7347f7a7527745e4da29c5/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:03.343919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6cb4e2-9441-44bf-b7a9-7d097dc2bda5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ca3316be8414d20996d0efd2dceee60, user_id=8bbbf977dd7347f7a7527745e4da29c5, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:03.345546 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6cb4e2-9441-44bf-b7a9-7d097dc2bda5 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:03.421016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:03.421697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d42ea-7d86-446c-92bc-2aabcba7a706 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:03.435542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 105/231] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:32:03 2022] PUT /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60/users/8bbbf977dd7347f7a7527745e4da29c5/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:03.448962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:03.462464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01f7c0-b8da-47b0-ac4f-7e4f18405151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:03.463157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01f7c0-b8da-47b0-ac4f-7e4f18405151 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:03.463447 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01f7c0-b8da-47b0-ac4f-7e4f18405151 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:03.463775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01f7c0-b8da-47b0-ac4f-7e4f18405151 None admin] PATH_INFO: `/v3/users/8bbbf977dd7347f7a7527745e4da29c5` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:03.465268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01f7c0-b8da-47b0-ac4f-7e4f18405151 None admin] RBAC: Authorizing `identity:get_user(user_id=8bbbf977dd7347f7a7527745e4da29c5)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:03.466909 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01f7c0-b8da-47b0-ac4f-7e4f18405151 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:03.468819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 127/232] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:03 2022] GET /identity/v3/users/8bbbf977dd7347f7a7527745e4da29c5 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:03.476059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cb4e2-9441-44bf-b7a9-7d097dc2bda5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:03.594779 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cb4e2-9441-44bf-b7a9-7d097dc2bda5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:03.632626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e1741-4263-4e96-a41c-6c2b9fb429fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:03.633309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e1741-4263-4e96-a41c-6c2b9fb429fc None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:03.633588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e1741-4263-4e96-a41c-6c2b9fb429fc None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:03.633922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e1741-4263-4e96-a41c-6c2b9fb429fc None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:03.635306 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e1741-4263-4e96-a41c-6c2b9fb429fc None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:03.636920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e1741-4263-4e96-a41c-6c2b9fb429fc None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:03.638619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 106/233] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:03 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:03.644598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01f7c0-b8da-47b0-ac4f-7e4f18405151 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:03.752636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01f7c0-b8da-47b0-ac4f-7e4f18405151 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:03.789956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254f491-adfe-43aa-9329-0ed27f3d8fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:03.790541 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254f491-adfe-43aa-9329-0ed27f3d8fb7 None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:03.790820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254f491-adfe-43aa-9329-0ed27f3d8fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:03.791095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254f491-adfe-43aa-9329-0ed27f3d8fb7 None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60/users/8bbbf977dd7347f7a7527745e4da29c5/roles/8183d3986255476f959797314695f92a` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:03.793285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254f491-adfe-43aa-9329-0ed27f3d8fb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ca3316be8414d20996d0efd2dceee60, user_id=8bbbf977dd7347f7a7527745e4da29c5, role_id=8183d3986255476f959797314695f92a)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:03.794786 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254f491-adfe-43aa-9329-0ed27f3d8fb7 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:03.911212 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 128/234] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:32:03 2022] PUT /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60/users/8bbbf977dd7347f7a7527745e4da29c5/roles/8183d3986255476f959797314695f92a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:04.080905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61edba1-1870-488b-83ea-894be54f7ef8 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:04.081442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61edba1-1870-488b-83ea-894be54f7ef8 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:04.081442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61edba1-1870-488b-83ea-894be54f7ef8 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:04.082969 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 107/235] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:04.102267 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d0dca-3f45-4678-8878-bd9fea582119 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:04.102569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d0dca-3f45-4678-8878-bd9fea582119 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:04.102814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d0dca-3f45-4678-8878-bd9fea582119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:04.111031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a78784-3d94-48e7-b423-bada4a6a013a None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:04.111312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a78784-3d94-48e7-b423-bada4a6a013a None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:04.111640 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a78784-3d94-48e7-b423-bada4a6a013a None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:04.113184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 108/236] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:32:04.123289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39835f5b-483f-49d5-97fb-ae471e5f1d52 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:04.123610 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39835f5b-483f-49d5-97fb-ae471e5f1d52 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:04.123844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39835f5b-483f-49d5-97fb-ae471e5f1d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:04.145347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-1b4d0dca-3f45-4678-8878-bd9fea582119 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:04.147411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-1b4d0dca-3f45-4678-8878-bd9fea582119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:04.156149 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-39835f5b-483f-49d5-97fb-ae471e5f1d52 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:04.157400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-39835f5b-483f-49d5-97fb-ae471e5f1d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:04.217905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:04.233815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:04.242099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 109/237] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:04.246645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 129/238] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:04.397250 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a9d92-bf0b-48c4-8d94-a5d99d50634c None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:04.397442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a9d92-bf0b-48c4-8d94-a5d99d50634c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:04.397716 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a9d92-bf0b-48c4-8d94-a5d99d50634c None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:04.399336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 110/239] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:04.404383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddf341-ca8a-423e-84bd-31c9b7d3532f None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:04.404694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddf341-ca8a-423e-84bd-31c9b7d3532f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:04.404977 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddf341-ca8a-423e-84bd-31c9b7d3532f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:04.429603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-4cddf341-ca8a-423e-84bd-31c9b7d3532f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:04.430800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4cddf341-ca8a-423e-84bd-31c9b7d3532f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:04.516886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 130/240] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:04.585423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a9d92-bf0b-48c4-8d94-a5d99d50634c None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:04.618651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddf341-ca8a-423e-84bd-31c9b7d3532f None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:04.695802 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a9d92-bf0b-48c4-8d94-a5d99d50634c None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:04.734500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7fb7f-51a1-4e59-ad4d-230d79622201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:04.735197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7fb7f-51a1-4e59-ad4d-230d79622201 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:04.735479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7fb7f-51a1-4e59-ad4d-230d79622201 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:04.735754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7fb7f-51a1-4e59-ad4d-230d79622201 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:04.741608 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e7fb7f-51a1-4e59-ad4d-230d79622201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:04.745580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e7fb7f-51a1-4e59-ad4d-230d79622201 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:04.747397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 111/241] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:04.753073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7fb7f-51a1-4e59-ad4d-230d79622201 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:04.813454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddf341-ca8a-423e-84bd-31c9b7d3532f None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:04.837003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:04.862034 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:04.878154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e183b06-bb4f-4de3-8711-ca018af67412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:04.879009 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e183b06-bb4f-4de3-8711-ca018af67412 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:04.880000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e183b06-bb4f-4de3-8711-ca018af67412 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:04.880509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e183b06-bb4f-4de3-8711-ca018af67412 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:04.894213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e183b06-bb4f-4de3-8711-ca018af67412 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-1e183b06-bb4f-4de3-8711-ca018af67412 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:04.902120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:04.903240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:04.904201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:04.904201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:04.904201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:04.904201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:04.904201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:04.904201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:04.904201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:04.904201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:04.904201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:04.904201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:04.904201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:32:04.904201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:04.906746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 131/242] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:04.907198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7fb7f-51a1-4e59-ad4d-230d79622201 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:04.907846 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e183b06-bb4f-4de3-8711-ca018af67412 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:04.951116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c431ea-ba44-4bb3-9bf9-2226e9271dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:04.951741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c431ea-ba44-4bb3-9bf9-2226e9271dc7 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:04.952090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c431ea-ba44-4bb3-9bf9-2226e9271dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:04.952691 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c431ea-ba44-4bb3-9bf9-2226e9271dc7 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:04.954276 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c431ea-ba44-4bb3-9bf9-2226e9271dc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:04.955789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c431ea-ba44-4bb3-9bf9-2226e9271dc7 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:05.025799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e183b06-bb4f-4de3-8711-ca018af67412 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:05.060987 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76174b-f740-45a4-acc0-516d19af935d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:05.061686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76174b-f740-45a4-acc0-516d19af935d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:05.061878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76174b-f740-45a4-acc0-516d19af935d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:05.062148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76174b-f740-45a4-acc0-516d19af935d None admin] PATH_INFO: `/v3/roles/member` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:05.066178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 112/243] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:32:04 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:32:05.067514 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c431ea-ba44-4bb3-9bf9-2226e9271dc7 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:05.074946 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76174b-f740-45a4-acc0-516d19af935d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-ff76174b-f740-45a4-acc0-516d19af935d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:05.080520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:05.081466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:05.082250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:05.082250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:05.082250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:05.082250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:05.082250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:05.082250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:05.082250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:05.082250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:05.082250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:05.082250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:05.082250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:05.082250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:05.082250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 132/244] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:05.090441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76174b-f740-45a4-acc0-516d19af935d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:05.127210 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:05.211951 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c431ea-ba44-4bb3-9bf9-2226e9271dc7 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:05.252986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9664b45-e736-4973-a92f-f8ee1ca3cc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:05.254959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9664b45-e736-4973-a92f-f8ee1ca3cc19 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:05.256575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9664b45-e736-4973-a92f-f8ee1ca3cc19 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:05.257178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9664b45-e736-4973-a92f-f8ee1ca3cc19 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:05.258237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9664b45-e736-4973-a92f-f8ee1ca3cc19 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:05.258770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9664b45-e736-4973-a92f-f8ee1ca3cc19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:05.260405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9664b45-e736-4973-a92f-f8ee1ca3cc19 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:05.268886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:05.272061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 113/245] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:05.277175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9664b45-e736-4973-a92f-f8ee1ca3cc19 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:05.282929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76174b-f740-45a4-acc0-516d19af935d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:05.327964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4804187-3be2-4de3-ab6f-68ad655debd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:05.328724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4804187-3be2-4de3-ab6f-68ad655debd3 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:05.329083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4804187-3be2-4de3-ab6f-68ad655debd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:05.329451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4804187-3be2-4de3-ab6f-68ad655debd3 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:05.330207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4804187-3be2-4de3-ab6f-68ad655debd3 None admin] RBAC: Adding query filter params (name=member) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:05.330583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4804187-3be2-4de3-ab6f-68ad655debd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:05.332278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4804187-3be2-4de3-ab6f-68ad655debd3 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:05.343703 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 133/246] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:05.349593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4804187-3be2-4de3-ab6f-68ad655debd3 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:05.415285 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9664b45-e736-4973-a92f-f8ee1ca3cc19 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:05.462764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:05.466497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a772f28-dafa-4552-9da4-3de4167fd307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:05.474714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a772f28-dafa-4552-9da4-3de4167fd307 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:05.474714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a772f28-dafa-4552-9da4-3de4167fd307 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:05.474714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a772f28-dafa-4552-9da4-3de4167fd307 None admin] PATH_INFO: `/v3/users/0155bbd7ac044a3da6bc9fad22cf6ad6` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:05.475180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4804187-3be2-4de3-ab6f-68ad655debd3 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:05.487994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a772f28-dafa-4552-9da4-3de4167fd307 None admin] RBAC: Authorizing `identity:get_user(user_id=0155bbd7ac044a3da6bc9fad22cf6ad6)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:05.490671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a772f28-dafa-4552-9da4-3de4167fd307 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:05.492971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 114/247] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:05 2022] GET /identity/v3/users/0155bbd7ac044a3da6bc9fad22cf6ad6 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:05.501296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a772f28-dafa-4552-9da4-3de4167fd307 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:05.519658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03d10a-527a-43eb-912f-4b2f103ac94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:05.520305 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03d10a-527a-43eb-912f-4b2f103ac94d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:05.520593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03d10a-527a-43eb-912f-4b2f103ac94d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:05.520987 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03d10a-527a-43eb-912f-4b2f103ac94d None admin] PATH_INFO: `/v3/users/b30c7335275449749c54c6485828c374` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:05.522481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03d10a-527a-43eb-912f-4b2f103ac94d None admin] RBAC: Authorizing `identity:get_user(user_id=b30c7335275449749c54c6485828c374)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:05.524401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03d10a-527a-43eb-912f-4b2f103ac94d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:05.527141 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 134/248] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:05 2022] GET /identity/v3/users/b30c7335275449749c54c6485828c374 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:05.533134 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03d10a-527a-43eb-912f-4b2f103ac94d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:05.561158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:05.625841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a772f28-dafa-4552-9da4-3de4167fd307 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:05.662927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea876046-b634-4e54-9a27-0ecb33c149ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:05.663551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea876046-b634-4e54-9a27-0ecb33c149ce None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:05.663834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea876046-b634-4e54-9a27-0ecb33c149ce None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:05.664105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea876046-b634-4e54-9a27-0ecb33c149ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:05.667208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03d10a-527a-43eb-912f-4b2f103ac94d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:05.707322 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d1b69-9588-4881-bd76-a0b8a908959b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:05.708372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d1b69-9588-4881-bd76-a0b8a908959b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:05.708938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d1b69-9588-4881-bd76-a0b8a908959b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:05.709415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d1b69-9588-4881-bd76-a0b8a908959b None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:05.710778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d1b69-9588-4881-bd76-a0b8a908959b None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:05.712430 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d1b69-9588-4881-bd76-a0b8a908959b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:05.714498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 135/249] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:05 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:05.715860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d357bc-bec8-4fc5-96d5-d443da61ef10 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:05.716343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d357bc-bec8-4fc5-96d5-d443da61ef10 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:05.716838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d357bc-bec8-4fc5-96d5-d443da61ef10 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:05.718580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 136/250] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:05.723668 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d357bc-bec8-4fc5-96d5-d443da61ef10 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:05.741239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea876046-b634-4e54-9a27-0ecb33c149ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:05.742752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea876046-b634-4e54-9a27-0ecb33c149ce None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:05.771737 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:05.797706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:05.868496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 115/251] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:05.872137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a17e0-de74-4e8e-82cb-1bd7309539ed None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:05.872137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a17e0-de74-4e8e-82cb-1bd7309539ed None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:05.872137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a17e0-de74-4e8e-82cb-1bd7309539ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:05.878371 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d357bc-bec8-4fc5-96d5-d443da61ef10 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:05.891882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-906a17e0-de74-4e8e-82cb-1bd7309539ed None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:05.893122 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-906a17e0-de74-4e8e-82cb-1bd7309539ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:05.929502 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25416d-8630-4854-aac1-83fc79105951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:05.930536 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25416d-8630-4854-aac1-83fc79105951 None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:05.931192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25416d-8630-4854-aac1-83fc79105951 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:05.932578 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25416d-8630-4854-aac1-83fc79105951 None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60/users/b30c7335275449749c54c6485828c374/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:05.934817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25416d-8630-4854-aac1-83fc79105951 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ca3316be8414d20996d0efd2dceee60, user_id=b30c7335275449749c54c6485828c374, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:05.936476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25416d-8630-4854-aac1-83fc79105951 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:05.972883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 116/252] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:05.974712 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a17e0-de74-4e8e-82cb-1bd7309539ed None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:06.027685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 137/253] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:32:05 2022] PUT /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60/users/b30c7335275449749c54c6485828c374/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:06.154379 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:06.193615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a17e0-de74-4e8e-82cb-1bd7309539ed None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:06.243485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71b718-d720-467c-8fbd-72bb3516275d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:06.244086 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71b718-d720-467c-8fbd-72bb3516275d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:06.244381 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71b718-d720-467c-8fbd-72bb3516275d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:06.244695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71b718-d720-467c-8fbd-72bb3516275d None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:06.245841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71b718-d720-467c-8fbd-72bb3516275d None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:06.247298 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71b718-d720-467c-8fbd-72bb3516275d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:06.249318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 117/254] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:05 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:06.254868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25416d-8630-4854-aac1-83fc79105951 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:06.382013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:06.395436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25416d-8630-4854-aac1-83fc79105951 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:06.396468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71b718-d720-467c-8fbd-72bb3516275d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:06.435139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f8c05-ce1f-46bf-b59d-307e984f01fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:06.435754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f8c05-ce1f-46bf-b59d-307e984f01fa None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:06.436037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f8c05-ce1f-46bf-b59d-307e984f01fa None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:06.439429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f8c05-ce1f-46bf-b59d-307e984f01fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:06.439429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f8c05-ce1f-46bf-b59d-307e984f01fa None admin] RBAC: Adding query filter params (role.id=27470a2fbc754a179323bb892496b013, scope.project.id=7ca3316be8414d20996d0efd2dceee60, user.id=0155bbd7ac044a3da6bc9fad22cf6ad6) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:06.439429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f8c05-ce1f-46bf-b59d-307e984f01fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:06.439733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f8c05-ce1f-46bf-b59d-307e984f01fa None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:06.446604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 138/255] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:32:06 2022] GET /identity/v3/role_assignments?user.id=0155bbd7ac044a3da6bc9fad22cf6ad6&scope.project.id=7ca3316be8414d20996d0efd2dceee60&role.id=27470a2fbc754a179323bb892496b013 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:06.575224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71b718-d720-467c-8fbd-72bb3516275d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:06.596850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffa4e3-7ad6-4c68-8e72-a788e158eefa None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:06.597153 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffa4e3-7ad6-4c68-8e72-a788e158eefa None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:06.597409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffa4e3-7ad6-4c68-8e72-a788e158eefa None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:06.600887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 139/256] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:06.611069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016691-2ee3-43b0-ae44-977cdf415271 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:06.611486 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016691-2ee3-43b0-ae44-977cdf415271 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:06.611767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016691-2ee3-43b0-ae44-977cdf415271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:06.626390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0deb8-11ea-4f50-9f4d-e82cc9fad350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:06.627411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0deb8-11ea-4f50-9f4d-e82cc9fad350 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:06.627902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0deb8-11ea-4f50-9f4d-e82cc9fad350 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:06.628374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0deb8-11ea-4f50-9f4d-e82cc9fad350 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:06.645697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:06.655315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-13016691-2ee3-43b0-ae44-977cdf415271 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:06.656528 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-13016691-2ee3-43b0-ae44-977cdf415271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:06.659379 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0deb8-11ea-4f50-9f4d-e82cc9fad350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-9ef0deb8-11ea-4f50-9f4d-e82cc9fad350 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:06.664284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:06.665223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:06.666027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:06.666027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:06.666027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:06.666027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:06.666027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:06.666027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:06.666027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:06.666027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:06.666027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:06.666027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:06.666027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:32:06.666027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:06.673847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 118/257] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:32:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:32:06.685631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0deb8-11ea-4f50-9f4d-e82cc9fad350 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:06.799953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 140/258] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:06.867046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0deb8-11ea-4f50-9f4d-e82cc9fad350 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:06.907199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13397969-6b9a-4831-928b-8f3ab4c9c2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:06.908222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13397969-6b9a-4831-928b-8f3ab4c9c2bc None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:06.908771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13397969-6b9a-4831-928b-8f3ab4c9c2bc None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:06.909272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13397969-6b9a-4831-928b-8f3ab4c9c2bc None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:06.910139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13397969-6b9a-4831-928b-8f3ab4c9c2bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:06.910643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13397969-6b9a-4831-928b-8f3ab4c9c2bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:06.912843 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13397969-6b9a-4831-928b-8f3ab4c9c2bc None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:06.922016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 119/259] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:06.927584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13016691-2ee3-43b0-ae44-977cdf415271 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:07.051764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:07.074292 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13016691-2ee3-43b0-ae44-977cdf415271 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:07.122264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9f17e-1756-495e-82da-ea04bec36579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:07.122925 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9f17e-1756-495e-82da-ea04bec36579 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:07.123207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9f17e-1756-495e-82da-ea04bec36579 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:07.123478 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9f17e-1756-495e-82da-ea04bec36579 None admin] PATH_INFO: `/v3/users/8bbbf977dd7347f7a7527745e4da29c5` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:07.124713 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9f17e-1756-495e-82da-ea04bec36579 None admin] RBAC: Authorizing `identity:get_user(user_id=8bbbf977dd7347f7a7527745e4da29c5)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:07.126273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9f17e-1756-495e-82da-ea04bec36579 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:07.128154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 141/260] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:06 2022] GET /identity/v3/users/8bbbf977dd7347f7a7527745e4da29c5 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:07.134299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13397969-6b9a-4831-928b-8f3ab4c9c2bc None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:07.207685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9f17e-1756-495e-82da-ea04bec36579 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:07.239447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:07.310335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13397969-6b9a-4831-928b-8f3ab4c9c2bc None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:07.366568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903307a-5609-465b-857a-f73796c0f0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:07.367762 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903307a-5609-465b-857a-f73796c0f0fa None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:07.368284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903307a-5609-465b-857a-f73796c0f0fa None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:07.368813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903307a-5609-465b-857a-f73796c0f0fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:07.382230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9f17e-1756-495e-82da-ea04bec36579 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:07.445356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce663ef-5fb6-45ad-a815-9be91a45c155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:07.445996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce663ef-5fb6-45ad-a815-9be91a45c155 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:07.446276 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce663ef-5fb6-45ad-a815-9be91a45c155 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:07.446544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce663ef-5fb6-45ad-a815-9be91a45c155 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:07.464454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903307a-5609-465b-857a-f73796c0f0fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:07.466231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903307a-5609-465b-857a-f73796c0f0fa None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:07.479481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce663ef-5fb6-45ad-a815-9be91a45c155 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-4ce663ef-5fb6-45ad-a815-9be91a45c155 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:07.484425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:07.485312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:07.486789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:07.486789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:07.486789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:07.486789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:07.486789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:07.486789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:07.486789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:07.486789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:07.486789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:07.486789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:07.486789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:07.486789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:07.486789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 142/261] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:32:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:32:07.493634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce663ef-5fb6-45ad-a815-9be91a45c155 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:07.512811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:07.575083 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:07.647389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 120/262] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:07.653121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903307a-5609-465b-857a-f73796c0f0fa None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:07.722644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce663ef-5fb6-45ad-a815-9be91a45c155 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:07.794646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903307a-5609-465b-857a-f73796c0f0fa None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:07.794870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff577f0e-f41a-4e36-a74f-58159cdff6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:07.795308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff577f0e-f41a-4e36-a74f-58159cdff6b3 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:07.795589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff577f0e-f41a-4e36-a74f-58159cdff6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:07.795866 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff577f0e-f41a-4e36-a74f-58159cdff6b3 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:07.796538 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff577f0e-f41a-4e36-a74f-58159cdff6b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:07.796888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff577f0e-f41a-4e36-a74f-58159cdff6b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:07.798283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff577f0e-f41a-4e36-a74f-58159cdff6b3 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:07.818939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 143/263] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:07.824406 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff577f0e-f41a-4e36-a74f-58159cdff6b3 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:07.833021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfea5e9-1c46-4a95-b6da-74208c8f84a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:07.834771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfea5e9-1c46-4a95-b6da-74208c8f84a6 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:07.835283 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfea5e9-1c46-4a95-b6da-74208c8f84a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:07.835749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfea5e9-1c46-4a95-b6da-74208c8f84a6 None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:07.837190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfea5e9-1c46-4a95-b6da-74208c8f84a6 None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:07.838847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfea5e9-1c46-4a95-b6da-74208c8f84a6 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:07.840687 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 121/264] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:07 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:07.846507 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfea5e9-1c46-4a95-b6da-74208c8f84a6 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:07.880174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:07.954400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfea5e9-1c46-4a95-b6da-74208c8f84a6 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:07.991414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7fb57-f42a-4e3c-ae49-1d1f026e4280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:07.992406 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7fb57-f42a-4e3c-ae49-1d1f026e4280 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:07.992931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7fb57-f42a-4e3c-ae49-1d1f026e4280 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:07.993415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7fb57-f42a-4e3c-ae49-1d1f026e4280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:07.994331 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7fb57-f42a-4e3c-ae49-1d1f026e4280 None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=7ca3316be8414d20996d0efd2dceee60, user.id=8bbbf977dd7347f7a7527745e4da29c5) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:07.994831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7fb57-f42a-4e3c-ae49-1d1f026e4280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:07.996502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7fb57-f42a-4e3c-ae49-1d1f026e4280 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:08.003169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 122/265] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:32:07 2022] GET /identity/v3/role_assignments?user.id=8bbbf977dd7347f7a7527745e4da29c5&scope.project.id=7ca3316be8414d20996d0efd2dceee60&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:08.038335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:08.065167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff577f0e-f41a-4e36-a74f-58159cdff6b3 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:08.114122 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c178fb7-fe45-4471-951e-fbef6831be74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:08.115295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c178fb7-fe45-4471-951e-fbef6831be74 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:08.115754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c178fb7-fe45-4471-951e-fbef6831be74 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:08.116231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c178fb7-fe45-4471-951e-fbef6831be74 None admin] PATH_INFO: `/v3/users/8bbbf977dd7347f7a7527745e4da29c5` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:08.117403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c178fb7-fe45-4471-951e-fbef6831be74 None admin] RBAC: Authorizing `identity:get_user(user_id=8bbbf977dd7347f7a7527745e4da29c5)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:08.118851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c178fb7-fe45-4471-951e-fbef6831be74 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:08.120418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 144/266] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:07 2022] GET /identity/v3/users/8bbbf977dd7347f7a7527745e4da29c5 => generated 283 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:08.126551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7fb57-f42a-4e3c-ae49-1d1f026e4280 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:08.222664 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bc03c-d5f8-4eef-9bbc-51cc3b451584 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:08.223237 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bc03c-d5f8-4eef-9bbc-51cc3b451584 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:08.223472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bc03c-d5f8-4eef-9bbc-51cc3b451584 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:08.225783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 145/267] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:08.232924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadcb57-eb4f-44aa-b25c-a5a2e70a4bb5 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:08.233222 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadcb57-eb4f-44aa-b25c-a5a2e70a4bb5 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:08.233484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadcb57-eb4f-44aa-b25c-a5a2e70a4bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:08.273072 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:08.275115 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-dbadcb57-eb4f-44aa-b25c-a5a2e70a4bb5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:08.276348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-dbadcb57-eb4f-44aa-b25c-a5a2e70a4bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:08.292895 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7fb57-f42a-4e3c-ae49-1d1f026e4280 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:08.385261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 146/268] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:32:08.386263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecd79c-f21e-483b-9b82-0562645e774c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:08.386888 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecd79c-f21e-483b-9b82-0562645e774c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:08.387167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecd79c-f21e-483b-9b82-0562645e774c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:08.387441 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecd79c-f21e-483b-9b82-0562645e774c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:08.500064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ecd79c-f21e-483b-9b82-0562645e774c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:08.501562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ecd79c-f21e-483b-9b82-0562645e774c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:08.609016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:08.619015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 123/269] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:08.625765 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbadcb57-eb4f-44aa-b25c-a5a2e70a4bb5 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:08.699548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecd79c-f21e-483b-9b82-0562645e774c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:08.723193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:08.759340 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbadcb57-eb4f-44aa-b25c-a5a2e70a4bb5 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:08.800233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487b1e3-8b22-4289-93e3-f3ad83a23ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:08.800894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487b1e3-8b22-4289-93e3-f3ad83a23ea5 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:08.801176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487b1e3-8b22-4289-93e3-f3ad83a23ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:08.801452 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487b1e3-8b22-4289-93e3-f3ad83a23ea5 None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:08.803135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487b1e3-8b22-4289-93e3-f3ad83a23ea5 None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:08.804598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487b1e3-8b22-4289-93e3-f3ad83a23ea5 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:08.806542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 147/270] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:08 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:08.807595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db138d67-e4df-4e5a-965d-fbcbf2e1e6d5 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:08.807885 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db138d67-e4df-4e5a-965d-fbcbf2e1e6d5 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:08.808191 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db138d67-e4df-4e5a-965d-fbcbf2e1e6d5 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:08.809386 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 148/271] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:08.814186 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db138d67-e4df-4e5a-965d-fbcbf2e1e6d5 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:08.862401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecd79c-f21e-483b-9b82-0562645e774c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:08.909491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099bb3d-66ff-4676-bf55-8864eacb4608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:08.910134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099bb3d-66ff-4676-bf55-8864eacb4608 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:08.910436 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099bb3d-66ff-4676-bf55-8864eacb4608 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:08.910697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099bb3d-66ff-4676-bf55-8864eacb4608 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:08.911883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099bb3d-66ff-4676-bf55-8864eacb4608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:08.913375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099bb3d-66ff-4676-bf55-8864eacb4608 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:08.915318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 124/272] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:08.916537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f729319-25dc-42ea-b456-301effa0be3c None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:08.916850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f729319-25dc-42ea-b456-301effa0be3c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:08.917179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f729319-25dc-42ea-b456-301effa0be3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:08.926479 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db138d67-e4df-4e5a-965d-fbcbf2e1e6d5 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:08.935700 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-0f729319-25dc-42ea-b456-301effa0be3c None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:08.936940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0f729319-25dc-42ea-b456-301effa0be3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:08.965279 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e378b8-d067-4b36-8e19-e3e37083532a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:08.965810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e378b8-d067-4b36-8e19-e3e37083532a None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:08.966092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e378b8-d067-4b36-8e19-e3e37083532a None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:08.966364 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e378b8-d067-4b36-8e19-e3e37083532a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:08.967088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e378b8-d067-4b36-8e19-e3e37083532a None admin] RBAC: Adding query filter params (role.id=8183d3986255476f959797314695f92a, scope.project.id=7ca3316be8414d20996d0efd2dceee60, user.id=8bbbf977dd7347f7a7527745e4da29c5) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:08.967401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e378b8-d067-4b36-8e19-e3e37083532a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:08.968849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e378b8-d067-4b36-8e19-e3e37083532a None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:08.976462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 149/273] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:32:08 2022] GET /identity/v3/role_assignments?user.id=8bbbf977dd7347f7a7527745e4da29c5&scope.project.id=7ca3316be8414d20996d0efd2dceee60&role.id=8183d3986255476f959797314695f92a => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:08.979456 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e378b8-d067-4b36-8e19-e3e37083532a None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:08.983431 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:09.009937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:09.020086 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 125/274] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:09.021710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db0790-42a7-4222-b1bd-a89622454b80 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:09.022005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db0790-42a7-4222-b1bd-a89622454b80 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:09.022280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db0790-42a7-4222-b1bd-a89622454b80 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:09.023888 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 126/275] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:09.029258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48dfda-26d0-449b-9c3e-2018f9196b57 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:09.029544 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48dfda-26d0-449b-9c3e-2018f9196b57 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:09.032004 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48dfda-26d0-449b-9c3e-2018f9196b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:09.075021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-2e48dfda-26d0-449b-9c3e-2018f9196b57 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:09.076261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2e48dfda-26d0-449b-9c3e-2018f9196b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:09.169921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e378b8-d067-4b36-8e19-e3e37083532a None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:09.171059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 127/276] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:09.213469 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc342da-123e-4d67-8fc3-1e9a0d9ca957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:09.214125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc342da-123e-4d67-8fc3-1e9a0d9ca957 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:09.214422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc342da-123e-4d67-8fc3-1e9a0d9ca957 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:09.214753 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc342da-123e-4d67-8fc3-1e9a0d9ca957 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:09.215462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc342da-123e-4d67-8fc3-1e9a0d9ca957 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:09.216888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc342da-123e-4d67-8fc3-1e9a0d9ca957 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:09.295414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48dfda-26d0-449b-9c3e-2018f9196b57 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:09.324502 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 150/277] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:32:08 2022] POST /identity/v3/users => generated 293 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:32:09.417649 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48dfda-26d0-449b-9c3e-2018f9196b57 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:09.435063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d3917-4ac0-49dd-99c7-980765d3ead1 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:09.435373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d3917-4ac0-49dd-99c7-980765d3ead1 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:09.435633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d3917-4ac0-49dd-99c7-980765d3ead1 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:09.437238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 151/278] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:09.441466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:09.443520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069757d-d20b-4a09-a19a-12b5e9f97224 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:09.443949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069757d-d20b-4a09-a19a-12b5e9f97224 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:09.444237 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069757d-d20b-4a09-a19a-12b5e9f97224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:09.458110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2657664-d712-42f7-8e54-2f59be4e855c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:09.459326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2657664-d712-42f7-8e54-2f59be4e855c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:09.459613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2657664-d712-42f7-8e54-2f59be4e855c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:09.459883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2657664-d712-42f7-8e54-2f59be4e855c None admin] PATH_INFO: `/v3/roles/member` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:09.464550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-3069757d-d20b-4a09-a19a-12b5e9f97224 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:09.466206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3069757d-d20b-4a09-a19a-12b5e9f97224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:09.471061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2657664-d712-42f7-8e54-2f59be4e855c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-f2657664-d712-42f7-8e54-2f59be4e855c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:09.477570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:09.478702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:09.479524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:09.479524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:09.479524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:09.479524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:09.479524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:09.479524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:09.479524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:09.479524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:09.479524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:09.479524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:09.479524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:09.479524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:09.482422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 128/279] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:32:09.483199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2657664-d712-42f7-8e54-2f59be4e855c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:09.530373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:09.545235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 152/280] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:32:09.553515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069757d-d20b-4a09-a19a-12b5e9f97224 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:09.609199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2657664-d712-42f7-8e54-2f59be4e855c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:09.655472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01117c9-c664-4c0d-acc0-9918e11d7534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:09.656120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01117c9-c664-4c0d-acc0-9918e11d7534 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:09.656402 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01117c9-c664-4c0d-acc0-9918e11d7534 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:09.656708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01117c9-c664-4c0d-acc0-9918e11d7534 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:09.668406 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01117c9-c664-4c0d-acc0-9918e11d7534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-e01117c9-c664-4c0d-acc0-9918e11d7534 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:09.673464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:09.674447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:09.675222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:09.675222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:09.675222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:09.675222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:09.675222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:09.675222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:09.675222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:09.675222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:09.675222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:09.675222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:09.675222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:09.675222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:09.675222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 129/281] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:09.681740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069757d-d20b-4a09-a19a-12b5e9f97224 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:09.682670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01117c9-c664-4c0d-acc0-9918e11d7534 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:09.722662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058605b-23d6-4626-ae92-3d4b6e7f66ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:09.723421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058605b-23d6-4626-ae92-3d4b6e7f66ae None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:09.723702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058605b-23d6-4626-ae92-3d4b6e7f66ae None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:09.724518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058605b-23d6-4626-ae92-3d4b6e7f66ae None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:09.725229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9058605b-23d6-4626-ae92-3d4b6e7f66ae None admin] RBAC: Adding query filter params (name=member) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:09.725537 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9058605b-23d6-4626-ae92-3d4b6e7f66ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:09.726971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9058605b-23d6-4626-ae92-3d4b6e7f66ae None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:09.731619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:09.737611 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 153/282] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:09.749178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058605b-23d6-4626-ae92-3d4b6e7f66ae None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:09.819217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01117c9-c664-4c0d-acc0-9918e11d7534 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:09.857159 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5d62c-ee3c-46c6-8a2c-f60ed8f4ac65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:09.857774 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5d62c-ee3c-46c6-8a2c-f60ed8f4ac65 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:09.858057 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5d62c-ee3c-46c6-8a2c-f60ed8f4ac65 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:09.858337 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5d62c-ee3c-46c6-8a2c-f60ed8f4ac65 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:09.859005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5d62c-ee3c-46c6-8a2c-f60ed8f4ac65 None admin] RBAC: Adding query filter params (name=member) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:09.859312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5d62c-ee3c-46c6-8a2c-f60ed8f4ac65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:09.860785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5d62c-ee3c-46c6-8a2c-f60ed8f4ac65 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:09.870220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:09.873035 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 130/283] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:09.879446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5d62c-ee3c-46c6-8a2c-f60ed8f4ac65 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:09.940890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058605b-23d6-4626-ae92-3d4b6e7f66ae None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:09.990413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac143111-a271-4f48-bf0f-7807141693d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:09.991505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac143111-a271-4f48-bf0f-7807141693d6 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:09.991858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac143111-a271-4f48-bf0f-7807141693d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:09.992207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac143111-a271-4f48-bf0f-7807141693d6 None admin] PATH_INFO: `/v3/users/cdb27c5fbe8c444c88de544f855ca70a` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:10.000946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5d62c-ee3c-46c6-8a2c-f60ed8f4ac65 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:10.002872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:10.015699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac143111-a271-4f48-bf0f-7807141693d6 None admin] RBAC: Authorizing `identity:get_user(user_id=cdb27c5fbe8c444c88de544f855ca70a)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:10.017492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac143111-a271-4f48-bf0f-7807141693d6 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:10.019324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 154/284] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:09 2022] GET /identity/v3/users/cdb27c5fbe8c444c88de544f855ca70a => generated 293 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:10.025510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac143111-a271-4f48-bf0f-7807141693d6 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:10.053585 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8e879-3738-4862-b88d-a7f6bea5a7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:10.053901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8e879-3738-4862-b88d-a7f6bea5a7c7 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:10.054155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8e879-3738-4862-b88d-a7f6bea5a7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:10.054442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8e879-3738-4862-b88d-a7f6bea5a7c7 None admin] PATH_INFO: `/v3/users/b30c7335275449749c54c6485828c374` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:10.055648 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8e879-3738-4862-b88d-a7f6bea5a7c7 None admin] RBAC: Authorizing `identity:get_user(user_id=b30c7335275449749c54c6485828c374)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:10.057118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8e879-3738-4862-b88d-a7f6bea5a7c7 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:10.058914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 131/285] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:09 2022] GET /identity/v3/users/b30c7335275449749c54c6485828c374 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:10.059475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8e879-3738-4862-b88d-a7f6bea5a7c7 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:10.144445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac143111-a271-4f48-bf0f-7807141693d6 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:10.154716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:10.180204 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8e879-3738-4862-b88d-a7f6bea5a7c7 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:10.181512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ae69c-e79a-4d63-a442-d3622f5cbc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:10.182685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ae69c-e79a-4d63-a442-d3622f5cbc19 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:10.183173 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ae69c-e79a-4d63-a442-d3622f5cbc19 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:10.183667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ae69c-e79a-4d63-a442-d3622f5cbc19 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:10.196377 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ae69c-e79a-4d63-a442-d3622f5cbc19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-a57ae69c-e79a-4d63-a442-d3622f5cbc19 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:10.201534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:10.202429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:10.203218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:10.203218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:10.203218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:10.203218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:10.203218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:10.203218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:10.203218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:10.203218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:10.203218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:10.203218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:10.203218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:32:10.203218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:10.205494 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 155/286] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:10.206712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ae69c-e79a-4d63-a442-d3622f5cbc19 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:10.218777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f70fd-77dc-44b7-b296-c9653ecd9dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:10.219422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f70fd-77dc-44b7-b296-c9653ecd9dc5 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:10.219704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f70fd-77dc-44b7-b296-c9653ecd9dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:10.219978 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f70fd-77dc-44b7-b296-c9653ecd9dc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:10.221115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f70fd-77dc-44b7-b296-c9653ecd9dc5 None admin] RBAC: Adding query filter params (role.id=9fe465cfe8764112a4cc92f3480326b8, user.id=cdb27c5fbe8c444c88de544f855ca70a, scope.system=all) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:10.221435 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f70fd-77dc-44b7-b296-c9653ecd9dc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:10.223450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f70fd-77dc-44b7-b296-c9653ecd9dc5 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:10.230488 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 132/287] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Aug 24 07:32:10 2022] GET /identity/v3/role_assignments?user.id=cdb27c5fbe8c444c88de544f855ca70a&scope.system=all&role.id=9fe465cfe8764112a4cc92f3480326b8 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:10.231617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f70fd-77dc-44b7-b296-c9653ecd9dc5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:10.321077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:10.332876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ae69c-e79a-4d63-a442-d3622f5cbc19 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:10.370607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f70fd-77dc-44b7-b296-c9653ecd9dc5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:10.414667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e456b2-45ce-4ac7-b573-91c0cfba2aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:10.415290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e456b2-45ce-4ac7-b573-91c0cfba2aeb None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:10.415569 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e456b2-45ce-4ac7-b573-91c0cfba2aeb None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:10.415841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e456b2-45ce-4ac7-b573-91c0cfba2aeb None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:10.416511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e456b2-45ce-4ac7-b573-91c0cfba2aeb None admin] RBAC: Adding query filter params (name=reader) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:10.416846 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e456b2-45ce-4ac7-b573-91c0cfba2aeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:10.418242 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e456b2-45ce-4ac7-b573-91c0cfba2aeb None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:10.421053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac1693-81c4-40ef-b35f-a29c1394bd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:10.421686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac1693-81c4-40ef-b35f-a29c1394bd8e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:10.421964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac1693-81c4-40ef-b35f-a29c1394bd8e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:10.422234 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac1693-81c4-40ef-b35f-a29c1394bd8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:10.428140 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 133/288] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:10.435091 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e456b2-45ce-4ac7-b573-91c0cfba2aeb None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:10.482891 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:10.522844 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac1693-81c4-40ef-b35f-a29c1394bd8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:10.524716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac1693-81c4-40ef-b35f-a29c1394bd8e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:10.551861 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:10.558719 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e456b2-45ce-4ac7-b573-91c0cfba2aeb None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:10.602420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc34056-eb8f-41c8-90e3-de9d7b0e808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:10.603177 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc34056-eb8f-41c8-90e3-de9d7b0e808d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:10.603557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc34056-eb8f-41c8-90e3-de9d7b0e808d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:10.605319 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc34056-eb8f-41c8-90e3-de9d7b0e808d None admin] PATH_INFO: `/v3/users/0155bbd7ac044a3da6bc9fad22cf6ad6` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:10.605930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc34056-eb8f-41c8-90e3-de9d7b0e808d None admin] RBAC: Authorizing `identity:get_user(user_id=0155bbd7ac044a3da6bc9fad22cf6ad6)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:10.607392 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc34056-eb8f-41c8-90e3-de9d7b0e808d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:10.612667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 134/289] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:10 2022] GET /identity/v3/users/0155bbd7ac044a3da6bc9fad22cf6ad6 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:10.616048 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc34056-eb8f-41c8-90e3-de9d7b0e808d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:10.731353 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc34056-eb8f-41c8-90e3-de9d7b0e808d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:10.772509 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef171e2-f0a4-4219-a95b-a5bb5e2e0ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:10.773189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef171e2-f0a4-4219-a95b-a5bb5e2e0ae4 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:10.773472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef171e2-f0a4-4219-a95b-a5bb5e2e0ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:10.773751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef171e2-f0a4-4219-a95b-a5bb5e2e0ae4 None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:10.777645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef171e2-f0a4-4219-a95b-a5bb5e2e0ae4 None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:10.779088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef171e2-f0a4-4219-a95b-a5bb5e2e0ae4 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:10.788624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 156/290] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:10.789525 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac1693-81c4-40ef-b35f-a29c1394bd8e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:10.790184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 135/291] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:10 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:10.793749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef171e2-f0a4-4219-a95b-a5bb5e2e0ae4 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:10.865385 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:10.941098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:10.980507 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef171e2-f0a4-4219-a95b-a5bb5e2e0ae4 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:11.021763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdc836-3587-448b-a2c7-7ae902742bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:11.022568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdc836-3587-448b-a2c7-7ae902742bfd None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:11.022853 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdc836-3587-448b-a2c7-7ae902742bfd None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:11.023127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdc836-3587-448b-a2c7-7ae902742bfd None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:11.024265 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdc836-3587-448b-a2c7-7ae902742bfd None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:11.026125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdc836-3587-448b-a2c7-7ae902742bfd None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:11.027985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 136/292] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:10 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:11.031516 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac1693-81c4-40ef-b35f-a29c1394bd8e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:11.033393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdc836-3587-448b-a2c7-7ae902742bfd None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:11.114076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f232883-1927-49f9-b981-63fc6315946e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:11.115373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f232883-1927-49f9-b981-63fc6315946e None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:11.115708 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f232883-1927-49f9-b981-63fc6315946e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:11.116014 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f232883-1927-49f9-b981-63fc6315946e None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60/users/0155bbd7ac044a3da6bc9fad22cf6ad6/roles/27470a2fbc754a179323bb892496b013` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:11.120187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f232883-1927-49f9-b981-63fc6315946e None admin] RBAC: Authorizing `identity:create_grant(project_id=7ca3316be8414d20996d0efd2dceee60, user_id=0155bbd7ac044a3da6bc9fad22cf6ad6, role_id=27470a2fbc754a179323bb892496b013)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:11.121725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f232883-1927-49f9-b981-63fc6315946e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:11.185793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdc836-3587-448b-a2c7-7ae902742bfd None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:11.188970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:11.207273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 157/293] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:32:10 2022] PUT /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60/users/0155bbd7ac044a3da6bc9fad22cf6ad6/roles/27470a2fbc754a179323bb892496b013 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:11.231017 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65add23-ab57-4815-a226-5fbf1af1f143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:11.231641 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65add23-ab57-4815-a226-5fbf1af1f143 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:11.231923 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65add23-ab57-4815-a226-5fbf1af1f143 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:11.232200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65add23-ab57-4815-a226-5fbf1af1f143 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:11.233035 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65add23-ab57-4815-a226-5fbf1af1f143 None admin] RBAC: Adding query filter params (role.id=9fe465cfe8764112a4cc92f3480326b8, scope.project.id=7ca3316be8414d20996d0efd2dceee60, user.id=b30c7335275449749c54c6485828c374) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:11.233346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65add23-ab57-4815-a226-5fbf1af1f143 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:11.234854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65add23-ab57-4815-a226-5fbf1af1f143 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:11.242345 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 137/294] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:32:11 2022] GET /identity/v3/role_assignments?user.id=b30c7335275449749c54c6485828c374&scope.project.id=7ca3316be8414d20996d0efd2dceee60&role.id=9fe465cfe8764112a4cc92f3480326b8 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:11.611597 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ee7a9-fca6-4148-860d-38ad626781ad None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:11.611937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ee7a9-fca6-4148-860d-38ad626781ad None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:11.612153 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ee7a9-fca6-4148-860d-38ad626781ad None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:11.613811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 158/295] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:11.619897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87d468-47bb-4eb3-aa93-da2cc9bbb400 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:11.620346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87d468-47bb-4eb3-aa93-da2cc9bbb400 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:11.620533 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87d468-47bb-4eb3-aa93-da2cc9bbb400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:11.640971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-6a87d468-47bb-4eb3-aa93-da2cc9bbb400 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:11.642175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6a87d468-47bb-4eb3-aa93-da2cc9bbb400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:11.680500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43854ba6-0a49-4ef2-a7f8-1f363eb82a0c None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:11.680852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43854ba6-0a49-4ef2-a7f8-1f363eb82a0c None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:11.681095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43854ba6-0a49-4ef2-a7f8-1f363eb82a0c None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:11.682587 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 159/296] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:11.688444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fee4fe-e5cc-4023-8733-53716b2f8619 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:11.688769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fee4fe-e5cc-4023-8733-53716b2f8619 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:11.689035 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fee4fe-e5cc-4023-8733-53716b2f8619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:11.716023 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-04fee4fe-e5cc-4023-8733-53716b2f8619 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:11.717628 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-04fee4fe-e5cc-4023-8733-53716b2f8619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:11.735499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 138/297] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:11.791924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 160/298] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:32:12.024502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87d468-47bb-4eb3-aa93-da2cc9bbb400 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:12.067254 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fee4fe-e5cc-4023-8733-53716b2f8619 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:12.124445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:12.132361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87d468-47bb-4eb3-aa93-da2cc9bbb400 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:12.140379 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:12.176391 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b91302-266f-48e0-ae29-919b6171b69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:12.177136 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b91302-266f-48e0-ae29-919b6171b69c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:12.177419 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b91302-266f-48e0-ae29-919b6171b69c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:12.177879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b91302-266f-48e0-ae29-919b6171b69c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:12.191157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b91302-266f-48e0-ae29-919b6171b69c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:12.196227 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fee4fe-e5cc-4023-8733-53716b2f8619 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-24b91302-266f-48e0-ae29-919b6171b69c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:12.197526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:12.198401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:12.199176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:12.199176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:12.199176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:12.199176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:12.199176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:12.199176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:12.199176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:12.199176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:12.199176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:12.199176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:12.199176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:32:12.199176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:12.199176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 139/299] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:12.204192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b91302-266f-48e0-ae29-919b6171b69c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:12.239389 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de37983-d4c9-4bcc-a92d-75145bd400fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:12.240102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de37983-d4c9-4bcc-a92d-75145bd400fd None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:12.240386 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de37983-d4c9-4bcc-a92d-75145bd400fd None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:12.240681 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de37983-d4c9-4bcc-a92d-75145bd400fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:12.241803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de37983-d4c9-4bcc-a92d-75145bd400fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:12.243266 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de37983-d4c9-4bcc-a92d-75145bd400fd None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:12.248703 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 161/300] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:12.251127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de37983-d4c9-4bcc-a92d-75145bd400fd None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:12.334268 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b91302-266f-48e0-ae29-919b6171b69c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:12.369918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de37983-d4c9-4bcc-a92d-75145bd400fd None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:12.372843 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3eeab-1f95-49c7-aa6e-2e17f9b263d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:12.373678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3eeab-1f95-49c7-aa6e-2e17f9b263d0 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:12.374155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3eeab-1f95-49c7-aa6e-2e17f9b263d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:12.374645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3eeab-1f95-49c7-aa6e-2e17f9b263d0 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:12.375556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3eeab-1f95-49c7-aa6e-2e17f9b263d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:12.376054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3eeab-1f95-49c7-aa6e-2e17f9b263d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:12.377761 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3eeab-1f95-49c7-aa6e-2e17f9b263d0 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:12.387417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 140/301] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:12.393622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3eeab-1f95-49c7-aa6e-2e17f9b263d0 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:12.403064 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:12.415903 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ff652-1a3e-4cdd-a1a1-11765aefd842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:12.417182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ff652-1a3e-4cdd-a1a1-11765aefd842 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:12.418048 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:12.418909 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ff652-1a3e-4cdd-a1a1-11765aefd842 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:12.419421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ff652-1a3e-4cdd-a1a1-11765aefd842 None admin] PATH_INFO: `/v3/groups` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:12.420251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ff652-1a3e-4cdd-a1a1-11765aefd842 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:12.422983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ff652-1a3e-4cdd-a1a1-11765aefd842 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:12.497589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 162/302] 10.222.0.26 () {60 vars in 1134 bytes} [Wed Aug 24 07:32:12 2022] POST /identity/v3/groups => generated 229 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:32:12.510216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3eeab-1f95-49c7-aa6e-2e17f9b263d0 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:12.552833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebb15e-98c6-4bca-ad6f-5cc0feb948f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:12.553494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebb15e-98c6-4bca-ad6f-5cc0feb948f6 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:12.553830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebb15e-98c6-4bca-ad6f-5cc0feb948f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:12.554200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebb15e-98c6-4bca-ad6f-5cc0feb948f6 None admin] PATH_INFO: `/v3/users/33a540304ffd43abb4f58cef11c804b3` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:12.570420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebb15e-98c6-4bca-ad6f-5cc0feb948f6 None admin] RBAC: Authorizing `identity:get_user(user_id=33a540304ffd43abb4f58cef11c804b3)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:12.571909 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebb15e-98c6-4bca-ad6f-5cc0feb948f6 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:12.573858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 141/303] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:12 2022] GET /identity/v3/users/33a540304ffd43abb4f58cef11c804b3 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:12.579993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ff652-1a3e-4cdd-a1a1-11765aefd842 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:12.580783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85df04-fe2f-4b9f-bdd6-691dedf0000b None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:12.581073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85df04-fe2f-4b9f-bdd6-691dedf0000b None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:12.581349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85df04-fe2f-4b9f-bdd6-691dedf0000b None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:12.582844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 142/304] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:32:12.589218 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026202b7-04c2-4446-b23a-0f4c71b16c12 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:12.589490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026202b7-04c2-4446-b23a-0f4c71b16c12 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:12.589696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026202b7-04c2-4446-b23a-0f4c71b16c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:12.609066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-026202b7-04c2-4446-b23a-0f4c71b16c12 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:12.610285 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-026202b7-04c2-4446-b23a-0f4c71b16c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:12.686460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 143/305] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:32:12.715641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ff652-1a3e-4cdd-a1a1-11765aefd842 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:12.752049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d340350-f3db-4566-b2eb-f481310fb61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:12.752682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d340350-f3db-4566-b2eb-f481310fb61f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:12.752969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d340350-f3db-4566-b2eb-f481310fb61f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:12.753243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d340350-f3db-4566-b2eb-f481310fb61f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:12.754077 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d340350-f3db-4566-b2eb-f481310fb61f None admin] RBAC: Adding query filter params (role.id=27470a2fbc754a179323bb892496b013, user.id=33a540304ffd43abb4f58cef11c804b3, scope.system=all) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:12.754393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d340350-f3db-4566-b2eb-f481310fb61f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:12.756638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d340350-f3db-4566-b2eb-f481310fb61f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:12.762103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 163/306] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Aug 24 07:32:12 2022] GET /identity/v3/role_assignments?user.id=33a540304ffd43abb4f58cef11c804b3&scope.system=all&role.id=27470a2fbc754a179323bb892496b013 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:12.963138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026202b7-04c2-4446-b23a-0f4c71b16c12 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:12.966270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:13.069693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026202b7-04c2-4446-b23a-0f4c71b16c12 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:13.106198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92088280-c484-476c-b4fd-3f25fb83134c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:13.106832 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088280-c484-476c-b4fd-3f25fb83134c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:13.107113 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088280-c484-476c-b4fd-3f25fb83134c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:13.107386 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088280-c484-476c-b4fd-3f25fb83134c None admin] PATH_INFO: `/v3/roles/member` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:13.118745 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92088280-c484-476c-b4fd-3f25fb83134c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-92088280-c484-476c-b4fd-3f25fb83134c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:13.123607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:13.124543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:13.125565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:13.125565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:13.125565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:13.125565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:13.125565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:13.125565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:13.125565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:13.125565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:13.125565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:13.125565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:13.125565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:13.125565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:13.125565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 144/307] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:13.130617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d340350-f3db-4566-b2eb-f481310fb61f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:13.183670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:13.237850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d340350-f3db-4566-b2eb-f481310fb61f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:13.276253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2d63a-9299-4d3d-ab48-478ffd6ebd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:13.277738 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2d63a-9299-4d3d-ab48-478ffd6ebd70 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:13.278224 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2d63a-9299-4d3d-ab48-478ffd6ebd70 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:13.278698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2d63a-9299-4d3d-ab48-478ffd6ebd70 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:13.279609 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2d63a-9299-4d3d-ab48-478ffd6ebd70 None admin] RBAC: Adding query filter params (name=member) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:13.280118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2d63a-9299-4d3d-ab48-478ffd6ebd70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:13.281730 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2d63a-9299-4d3d-ab48-478ffd6ebd70 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:13.291178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 164/308] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:13.296919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92088280-c484-476c-b4fd-3f25fb83134c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:13.306518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca440fe4-3ad7-40d9-917f-2d248d0103d6 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:13.306806 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca440fe4-3ad7-40d9-917f-2d248d0103d6 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:13.307081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca440fe4-3ad7-40d9-917f-2d248d0103d6 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:13.308399 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 165/309] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:32:13.313200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24876fb-4e7c-47e0-8598-a22fbe4043c0 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:13.313482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24876fb-4e7c-47e0-8598-a22fbe4043c0 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:13.313755 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24876fb-4e7c-47e0-8598-a22fbe4043c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:13.336613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-d24876fb-4e7c-47e0-8598-a22fbe4043c0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:13.337855 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d24876fb-4e7c-47e0-8598-a22fbe4043c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:13.372243 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:13.412473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92088280-c484-476c-b4fd-3f25fb83134c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:13.435011 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 166/310] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:32:13.455259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10be37d-d3a3-4137-a27f-f125f3996e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:13.455940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10be37d-d3a3-4137-a27f-f125f3996e0c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:13.456223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10be37d-d3a3-4137-a27f-f125f3996e0c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:13.456495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10be37d-d3a3-4137-a27f-f125f3996e0c None admin] PATH_INFO: `/v3/users/cdb27c5fbe8c444c88de544f855ca70a` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:13.457654 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10be37d-d3a3-4137-a27f-f125f3996e0c None admin] RBAC: Authorizing `identity:get_user(user_id=cdb27c5fbe8c444c88de544f855ca70a)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:13.459319 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10be37d-d3a3-4137-a27f-f125f3996e0c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:13.460908 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 145/311] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:13 2022] GET /identity/v3/users/cdb27c5fbe8c444c88de544f855ca70a => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:13.466481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24876fb-4e7c-47e0-8598-a22fbe4043c0 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:13.511786 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:13.574904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24876fb-4e7c-47e0-8598-a22fbe4043c0 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:13.610391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e82918-4a27-4f3d-902f-35db990f5f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:13.611046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e82918-4a27-4f3d-902f-35db990f5f99 None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:13.611324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e82918-4a27-4f3d-902f-35db990f5f99 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:13.611602 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e82918-4a27-4f3d-902f-35db990f5f99 None admin] PATH_INFO: `/v3/system/users/cdb27c5fbe8c444c88de544f855ca70a/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:13.613022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e82918-4a27-4f3d-902f-35db990f5f99 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cdb27c5fbe8c444c88de544f855ca70a, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:13.614568 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e82918-4a27-4f3d-902f-35db990f5f99 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:13.722298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 167/312] 10.222.0.26 () {58 vars in 1334 bytes} [Wed Aug 24 07:32:13 2022] PUT /identity/v3/system/users/cdb27c5fbe8c444c88de544f855ca70a/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:13.733201 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10be37d-d3a3-4137-a27f-f125f3996e0c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:13.850944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10be37d-d3a3-4137-a27f-f125f3996e0c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:13.885286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:13.890642 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf39b7-f3af-447b-a9f6-a96c72e01285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:13.891292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf39b7-f3af-447b-a9f6-a96c72e01285 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:13.891568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf39b7-f3af-447b-a9f6-a96c72e01285 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:13.902789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf39b7-f3af-447b-a9f6-a96c72e01285 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:13.902789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf39b7-f3af-447b-a9f6-a96c72e01285 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-f1cf39b7-f3af-447b-a9f6-a96c72e01285 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:13.907416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:13.908531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:13.909468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:13.909468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:13.909468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:13.909468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:13.909468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:13.909468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:13.909468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:13.909468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:13.909468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:13.909468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:13.909468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:32:13.909468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:13.909468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 146/313] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:13.916470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e82918-4a27-4f3d-902f-35db990f5f99 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:14.024140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e82918-4a27-4f3d-902f-35db990f5f99 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:14.031471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:14.060042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a9bee-d4cb-44e9-a151-7056a673e2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:14.060671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a9bee-d4cb-44e9-a151-7056a673e2da None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:14.060972 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a9bee-d4cb-44e9-a151-7056a673e2da None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:14.061250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a9bee-d4cb-44e9-a151-7056a673e2da None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:14.061921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a9bee-d4cb-44e9-a151-7056a673e2da None admin] RBAC: Adding query filter params (name=reader) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:14.062229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a9bee-d4cb-44e9-a151-7056a673e2da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:14.063642 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a9bee-d4cb-44e9-a151-7056a673e2da None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:14.072736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 168/314] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:14.078827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf39b7-f3af-447b-a9f6-a96c72e01285 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:14.187668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf39b7-f3af-447b-a9f6-a96c72e01285 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:14.224636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627bcf6-ccc3-4b55-9986-afc23bbb126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:14.225669 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627bcf6-ccc3-4b55-9986-afc23bbb126e None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:14.226164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627bcf6-ccc3-4b55-9986-afc23bbb126e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:14.226627 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627bcf6-ccc3-4b55-9986-afc23bbb126e None admin] PATH_INFO: `/v3/users/0155bbd7ac044a3da6bc9fad22cf6ad6` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:14.227955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627bcf6-ccc3-4b55-9986-afc23bbb126e None admin] RBAC: Authorizing `identity:get_user(user_id=0155bbd7ac044a3da6bc9fad22cf6ad6)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:14.229629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627bcf6-ccc3-4b55-9986-afc23bbb126e None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:14.231730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 147/315] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:14 2022] GET /identity/v3/users/0155bbd7ac044a3da6bc9fad22cf6ad6 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:14.238120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a9bee-d4cb-44e9-a151-7056a673e2da None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:14.352019 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a9bee-d4cb-44e9-a151-7056a673e2da None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:14.387053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac003fba-c4bc-49f7-83bd-464b07efa62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:14.387669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac003fba-c4bc-49f7-83bd-464b07efa62a None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:14.387952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac003fba-c4bc-49f7-83bd-464b07efa62a None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:14.388229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac003fba-c4bc-49f7-83bd-464b07efa62a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:14.424487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:14.460759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac003fba-c4bc-49f7-83bd-464b07efa62a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:14.460759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac003fba-c4bc-49f7-83bd-464b07efa62a None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:14.563868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 169/316] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:14.571844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627bcf6-ccc3-4b55-9986-afc23bbb126e None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:14.615742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d20dd8-172e-4a95-83e4-55ae57076c37 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:14.616048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d20dd8-172e-4a95-83e4-55ae57076c37 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:14.616289 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d20dd8-172e-4a95-83e4-55ae57076c37 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:14.617930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 170/317] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:14.623970 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a68e8-4639-4d0c-934a-d1cfdd5d05fb None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:14.624251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a68e8-4639-4d0c-934a-d1cfdd5d05fb None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:14.624525 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a68e8-4639-4d0c-934a-d1cfdd5d05fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:14.642732 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-823a68e8-4639-4d0c-934a-d1cfdd5d05fb None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:14.643928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-823a68e8-4639-4d0c-934a-d1cfdd5d05fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:14.647562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:14.696679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627bcf6-ccc3-4b55-9986-afc23bbb126e None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:14.709433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:14.728030 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 171/318] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:32:14.735741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32263483-2e42-42f0-ace1-89e0cd4c8dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:14.736482 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32263483-2e42-42f0-ace1-89e0cd4c8dd8 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:14.736892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32263483-2e42-42f0-ace1-89e0cd4c8dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:14.737933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32263483-2e42-42f0-ace1-89e0cd4c8dd8 None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:14.739214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32263483-2e42-42f0-ace1-89e0cd4c8dd8 None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:14.741514 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32263483-2e42-42f0-ace1-89e0cd4c8dd8 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:14.743630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 148/319] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:14 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:14.749088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a68e8-4639-4d0c-934a-d1cfdd5d05fb None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:14.773971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb294f4-dbd2-4553-8406-e43fb8b333ae None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:14.774257 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb294f4-dbd2-4553-8406-e43fb8b333ae None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:14.774537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb294f4-dbd2-4553-8406-e43fb8b333ae None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:14.776381 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 149/320] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:32:14.781020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78fb96-3118-4ab2-bd89-9910ea096487 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:14.781311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78fb96-3118-4ab2-bd89-9910ea096487 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:14.781589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78fb96-3118-4ab2-bd89-9910ea096487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:14.800893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-2e78fb96-3118-4ab2-bd89-9910ea096487 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:14.803021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2e78fb96-3118-4ab2-bd89-9910ea096487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:14.859049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a68e8-4639-4d0c-934a-d1cfdd5d05fb None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:14.893170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 150/321] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:14.897093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc146e2-3c14-4ee3-8b1c-b71d4dab67ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:14.897874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc146e2-3c14-4ee3-8b1c-b71d4dab67ab None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:14.898288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc146e2-3c14-4ee3-8b1c-b71d4dab67ab None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:14.898717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc146e2-3c14-4ee3-8b1c-b71d4dab67ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:14.899571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc146e2-3c14-4ee3-8b1c-b71d4dab67ab None admin] RBAC: Adding query filter params (role.id=27470a2fbc754a179323bb892496b013, scope.project.id=7ca3316be8414d20996d0efd2dceee60, user.id=0155bbd7ac044a3da6bc9fad22cf6ad6) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:14.900009 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc146e2-3c14-4ee3-8b1c-b71d4dab67ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:14.901605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc146e2-3c14-4ee3-8b1c-b71d4dab67ab None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:14.910953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 172/322] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:32:14 2022] GET /identity/v3/role_assignments?user.id=0155bbd7ac044a3da6bc9fad22cf6ad6&scope.project.id=7ca3316be8414d20996d0efd2dceee60&role.id=27470a2fbc754a179323bb892496b013 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:15.030451 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78fb96-3118-4ab2-bd89-9910ea096487 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:15.096022 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:15.140367 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78fb96-3118-4ab2-bd89-9910ea096487 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:15.173442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc146e2-3c14-4ee3-8b1c-b71d4dab67ab None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:15.179842 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a899e-ed05-4d5b-b371-dc87dbf8c01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:15.180481 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a899e-ed05-4d5b-b371-dc87dbf8c01e None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:15.180796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a899e-ed05-4d5b-b371-dc87dbf8c01e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:15.183871 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a899e-ed05-4d5b-b371-dc87dbf8c01e None admin] PATH_INFO: `/v3/domains/default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:15.184496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a899e-ed05-4d5b-b371-dc87dbf8c01e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:15.186132 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a899e-ed05-4d5b-b371-dc87dbf8c01e None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:15.188476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 151/323] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:15.194020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a899e-ed05-4d5b-b371-dc87dbf8c01e None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:15.242341 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:15.282707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc146e2-3c14-4ee3-8b1c-b71d4dab67ab None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:15.301008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a899e-ed05-4d5b-b371-dc87dbf8c01e None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:15.319344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9242095-aa20-4475-9189-5d5e6bd788aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:15.319979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9242095-aa20-4475-9189-5d5e6bd788aa None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:15.320261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9242095-aa20-4475-9189-5d5e6bd788aa None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:15.320535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9242095-aa20-4475-9189-5d5e6bd788aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:15.331510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9242095-aa20-4475-9189-5d5e6bd788aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:15.331648 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-d9242095-aa20-4475-9189-5d5e6bd788aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:15.337052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:15.341518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:15.342308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:15.342308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:15.342308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:15.342308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:15.342308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:15.342308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:15.342308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:15.342308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:15.342308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:15.342308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:15.342308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:32:15.342308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:15.342308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 173/324] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:32:15.343158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de123838-cfde-48c7-a933-e9dbf0a5fb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:15.343158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de123838-cfde-48c7-a933-e9dbf0a5fb8a None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:15.343158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de123838-cfde-48c7-a933-e9dbf0a5fb8a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:15.343158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de123838-cfde-48c7-a933-e9dbf0a5fb8a None admin] PATH_INFO: `/v3/groups` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:15.343412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de123838-cfde-48c7-a933-e9dbf0a5fb8a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:15.344408 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de123838-cfde-48c7-a933-e9dbf0a5fb8a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:15.344501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9242095-aa20-4475-9189-5d5e6bd788aa None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:15.446618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 152/325] 10.222.0.26 () {60 vars in 1134 bytes} [Wed Aug 24 07:32:15 2022] POST /identity/v3/groups => generated 226 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:32:15.447779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9242095-aa20-4475-9189-5d5e6bd788aa None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:15.479039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:15.491836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87db8d-60d6-42a8-85e2-9232898f99a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:15.492465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87db8d-60d6-42a8-85e2-9232898f99a8 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:15.492784 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87db8d-60d6-42a8-85e2-9232898f99a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:15.493062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87db8d-60d6-42a8-85e2-9232898f99a8 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:15.493752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87db8d-60d6-42a8-85e2-9232898f99a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:15.494059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87db8d-60d6-42a8-85e2-9232898f99a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:15.495510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87db8d-60d6-42a8-85e2-9232898f99a8 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:15.506169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 174/326] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:15.511957 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de123838-cfde-48c7-a933-e9dbf0a5fb8a None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:15.622284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de123838-cfde-48c7-a933-e9dbf0a5fb8a None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:15.657583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2b41a-2437-4469-82f1-d7b7ce901d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:15.658221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2b41a-2437-4469-82f1-d7b7ce901d3f None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:15.658581 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2b41a-2437-4469-82f1-d7b7ce901d3f None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:15.658878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2b41a-2437-4469-82f1-d7b7ce901d3f None admin] PATH_INFO: `/v3/users/33a540304ffd43abb4f58cef11c804b3` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:15.659995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2b41a-2437-4469-82f1-d7b7ce901d3f None admin] RBAC: Authorizing `identity:get_user(user_id=33a540304ffd43abb4f58cef11c804b3)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:15.662012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2b41a-2437-4469-82f1-d7b7ce901d3f None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:15.663743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 153/327] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:15 2022] GET /identity/v3/users/33a540304ffd43abb4f58cef11c804b3 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:15.669310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87db8d-60d6-42a8-85e2-9232898f99a8 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:15.775528 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87db8d-60d6-42a8-85e2-9232898f99a8 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:15.788863 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d20c1-b3c6-4aee-b906-e7adeb666f76 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:15.789381 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d20c1-b3c6-4aee-b906-e7adeb666f76 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:15.789381 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d20c1-b3c6-4aee-b906-e7adeb666f76 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:15.790910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 154/328] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:15.797530 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619faf83-d59b-474a-be4e-14840b54ba83 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:15.797812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619faf83-d59b-474a-be4e-14840b54ba83 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:15.798081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619faf83-d59b-474a-be4e-14840b54ba83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:15.813804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22bd9d-75db-4153-a85d-ce76d52b4c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:15.814453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22bd9d-75db-4153-a85d-ce76d52b4c76 None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:15.814731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22bd9d-75db-4153-a85d-ce76d52b4c76 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:15.815003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22bd9d-75db-4153-a85d-ce76d52b4c76 None admin] PATH_INFO: `/v3/system/users/33a540304ffd43abb4f58cef11c804b3/roles/27470a2fbc754a179323bb892496b013` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:15.816406 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22bd9d-75db-4153-a85d-ce76d52b4c76 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=33a540304ffd43abb4f58cef11c804b3, role_id=27470a2fbc754a179323bb892496b013)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:15.818023 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22bd9d-75db-4153-a85d-ce76d52b4c76 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:15.820346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-619faf83-d59b-474a-be4e-14840b54ba83 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:15.821592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-619faf83-d59b-474a-be4e-14840b54ba83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:15.887252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:15.893307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 175/329] 10.222.0.26 () {58 vars in 1334 bytes} [Wed Aug 24 07:32:15 2022] PUT /identity/v3/system/users/33a540304ffd43abb4f58cef11c804b3/roles/27470a2fbc754a179323bb892496b013 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:15.907315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 155/330] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:16.193772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22bd9d-75db-4153-a85d-ce76d52b4c76 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:16.253361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:16.316381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22bd9d-75db-4153-a85d-ce76d52b4c76 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:16.361683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3efcde-6efb-4654-b63a-22088f8f865e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:16.362313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3efcde-6efb-4654-b63a-22088f8f865e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:16.362589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3efcde-6efb-4654-b63a-22088f8f865e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:16.362859 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3efcde-6efb-4654-b63a-22088f8f865e None admin] PATH_INFO: `/v3/roles/member` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:16.377137 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3efcde-6efb-4654-b63a-22088f8f865e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-fb3efcde-6efb-4654-b63a-22088f8f865e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:16.382589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:16.383462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:16.384236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:16.384236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:16.384236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:16.384236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:16.384236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:16.384236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:16.384236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:16.384236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:16.384236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:16.384236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:16.384236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:16.384236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:16.384236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 176/331] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:16.389614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619faf83-d59b-474a-be4e-14840b54ba83 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:16.523254 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619faf83-d59b-474a-be4e-14840b54ba83 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:16.573607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc739e0-3c13-4eee-9a9c-841be5f2dbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:16.574222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc739e0-3c13-4eee-9a9c-841be5f2dbb0 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:16.574502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc739e0-3c13-4eee-9a9c-841be5f2dbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:16.574814 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc739e0-3c13-4eee-9a9c-841be5f2dbb0 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:16.575483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc739e0-3c13-4eee-9a9c-841be5f2dbb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:16.575785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc739e0-3c13-4eee-9a9c-841be5f2dbb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:16.577229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc739e0-3c13-4eee-9a9c-841be5f2dbb0 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:16.588433 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 156/332] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:16.595041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3efcde-6efb-4654-b63a-22088f8f865e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:16.750766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3efcde-6efb-4654-b63a-22088f8f865e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:16.766709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:16.799354 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadf376-8ef8-485c-9dcb-b8508dffa362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:16.799915 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadf376-8ef8-485c-9dcb-b8508dffa362 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:16.800198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadf376-8ef8-485c-9dcb-b8508dffa362 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:16.800474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadf376-8ef8-485c-9dcb-b8508dffa362 None admin] PATH_INFO: `/v3/users/cdb27c5fbe8c444c88de544f855ca70a` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:16.805359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadf376-8ef8-485c-9dcb-b8508dffa362 None admin] RBAC: Authorizing `identity:get_user(user_id=cdb27c5fbe8c444c88de544f855ca70a)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:16.806831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadf376-8ef8-485c-9dcb-b8508dffa362 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:16.808819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 177/333] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:16 2022] GET /identity/v3/users/cdb27c5fbe8c444c88de544f855ca70a => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:16.814394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc739e0-3c13-4eee-9a9c-841be5f2dbb0 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:16.921581 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:16.976919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc739e0-3c13-4eee-9a9c-841be5f2dbb0 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:17.017303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36744caa-db55-48c8-a25c-f5700e665a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:17.017926 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36744caa-db55-48c8-a25c-f5700e665a76 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:17.018212 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36744caa-db55-48c8-a25c-f5700e665a76 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:17.018492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36744caa-db55-48c8-a25c-f5700e665a76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:17.019225 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36744caa-db55-48c8-a25c-f5700e665a76 None admin] RBAC: Adding query filter params (role.id=9fe465cfe8764112a4cc92f3480326b8, user.id=cdb27c5fbe8c444c88de544f855ca70a, scope.system=all) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:17.019536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36744caa-db55-48c8-a25c-f5700e665a76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:17.021546 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36744caa-db55-48c8-a25c-f5700e665a76 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:17.036580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 157/334] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Aug 24 07:32:16 2022] GET /identity/v3/role_assignments?user.id=cdb27c5fbe8c444c88de544f855ca70a&scope.system=all&role.id=9fe465cfe8764112a4cc92f3480326b8 => generated 510 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:17.779875 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca2652-2a8b-4685-8f7b-0cf98985292f None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:17.780386 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca2652-2a8b-4685-8f7b-0cf98985292f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:17.780490 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca2652-2a8b-4685-8f7b-0cf98985292f None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:17.782111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 178/335] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:17.787918 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14dd6f-d24f-41e0-b4f0-0d55ba2f549b None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:17.788214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14dd6f-d24f-41e0-b4f0-0d55ba2f549b None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:17.788499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14dd6f-d24f-41e0-b4f0-0d55ba2f549b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:17.812690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-ee14dd6f-d24f-41e0-b4f0-0d55ba2f549b None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:17.813899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ee14dd6f-d24f-41e0-b4f0-0d55ba2f549b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:17.902813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 158/336] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:32:18.233966 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca2652-2a8b-4685-8f7b-0cf98985292f None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:18.273281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fccef2-dc2f-42d6-bfe3-b70180c4d7b8 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:18.273619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fccef2-dc2f-42d6-bfe3-b70180c4d7b8 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:18.273838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fccef2-dc2f-42d6-bfe3-b70180c4d7b8 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:18.275423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 159/337] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:32:18.301562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a778f-d0e1-46fe-b119-540b15384985 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:18.302051 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a778f-d0e1-46fe-b119-540b15384985 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:18.302334 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a778f-d0e1-46fe-b119-540b15384985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:18.334397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-574a778f-d0e1-46fe-b119-540b15384985 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:18.335622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-574a778f-d0e1-46fe-b119-540b15384985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:18.355836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca2652-2a8b-4685-8f7b-0cf98985292f None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:18.384417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:18.443204 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 160/338] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:18.449937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbca405-6da3-40fd-84f5-c64a5669476f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:18.450720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbca405-6da3-40fd-84f5-c64a5669476f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:18.451101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbca405-6da3-40fd-84f5-c64a5669476f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:18.451475 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbca405-6da3-40fd-84f5-c64a5669476f None admin] PATH_INFO: `/v3/roles/member` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:18.476438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbca405-6da3-40fd-84f5-c64a5669476f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-adbca405-6da3-40fd-84f5-c64a5669476f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:18.485515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:18.486392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:18.487241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:18.487241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:18.487241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:18.487241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:18.487241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:18.487241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:18.487241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:18.487241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:18.487241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:18.487241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:18.487241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:18.487241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:18.487241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 179/339] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:18.492938 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a778f-d0e1-46fe-b119-540b15384985 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:18.641782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a778f-d0e1-46fe-b119-540b15384985 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:18.685493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea21753-dfc4-45be-adda-1ac47367b239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:18.686460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea21753-dfc4-45be-adda-1ac47367b239 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:18.686938 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea21753-dfc4-45be-adda-1ac47367b239 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:18.687407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea21753-dfc4-45be-adda-1ac47367b239 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:18.688287 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea21753-dfc4-45be-adda-1ac47367b239 None admin] RBAC: Adding query filter params (name=member) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:18.688816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea21753-dfc4-45be-adda-1ac47367b239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:18.690379 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea21753-dfc4-45be-adda-1ac47367b239 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:18.699121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 161/340] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:18.704786 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbca405-6da3-40fd-84f5-c64a5669476f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:18.746181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:18.813582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbca405-6da3-40fd-84f5-c64a5669476f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:18.855702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ababf7-c853-45b7-b145-f47332515c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:18.856680 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ababf7-c853-45b7-b145-f47332515c17 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:18.857162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ababf7-c853-45b7-b145-f47332515c17 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:18.857661 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ababf7-c853-45b7-b145-f47332515c17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:18.934069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ababf7-c853-45b7-b145-f47332515c17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:18.935871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ababf7-c853-45b7-b145-f47332515c17 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:18.981500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:19.003043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea21753-dfc4-45be-adda-1ac47367b239 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:19.054312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:19.081509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 180/341] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:19.114756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ababf7-c853-45b7-b145-f47332515c17 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:19.201967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea21753-dfc4-45be-adda-1ac47367b239 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:19.237752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a40e5-bf8f-4281-a228-37e7bce2c638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:19.238728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a40e5-bf8f-4281-a228-37e7bce2c638 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:19.239774 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a40e5-bf8f-4281-a228-37e7bce2c638 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:19.240249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a40e5-bf8f-4281-a228-37e7bce2c638 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:19.252269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a40e5-bf8f-4281-a228-37e7bce2c638 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-b60a40e5-bf8f-4281-a228-37e7bce2c638 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:19.258100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:19.258960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:19.259735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:19.259735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:19.259735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:19.259735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:19.259735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:19.259735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:19.259735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:19.259735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:19.259735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:19.259735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:19.259735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:19.259735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:19.262410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 162/342] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:32:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:19.264354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588da475-7c81-4ae6-9b93-c626ed197b4a None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:19.264886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588da475-7c81-4ae6-9b93-c626ed197b4a None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:19.265086 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ababf7-c853-45b7-b145-f47332515c17 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:19.265553 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588da475-7c81-4ae6-9b93-c626ed197b4a None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:19.267105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 163/343] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:19.268337 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953a031-8bd2-41b0-bb30-ed7a6961913c None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:19.272927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953a031-8bd2-41b0-bb30-ed7a6961913c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:19.273436 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953a031-8bd2-41b0-bb30-ed7a6961913c None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:19.274914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 164/344] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:19.275854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953a031-8bd2-41b0-bb30-ed7a6961913c None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:19.311632 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346c734-0efb-4cbf-9280-f39da83ac19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:19.312278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346c734-0efb-4cbf-9280-f39da83ac19e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:19.312561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346c734-0efb-4cbf-9280-f39da83ac19e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:19.312866 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346c734-0efb-4cbf-9280-f39da83ac19e None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:19.314564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346c734-0efb-4cbf-9280-f39da83ac19e None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:19.316030 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346c734-0efb-4cbf-9280-f39da83ac19e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:19.318008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 181/345] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:19 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:19.319059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d80fe-1367-4f2f-9bef-433c8a0ab92e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:19.319344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d80fe-1367-4f2f-9bef-433c8a0ab92e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:19.319774 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d80fe-1367-4f2f-9bef-433c8a0ab92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:19.332106 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:19.346038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-629d80fe-1367-4f2f-9bef-433c8a0ab92e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:19.347258 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-629d80fe-1367-4f2f-9bef-433c8a0ab92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:19.381824 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:19.403965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953a031-8bd2-41b0-bb30-ed7a6961913c None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:19.446520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 182/346] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:19.446729 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215527c-5940-47cf-aae9-5130d1c91e4a None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:19.446935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215527c-5940-47cf-aae9-5130d1c91e4a None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:19.447197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215527c-5940-47cf-aae9-5130d1c91e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:19.450154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449dc14e-545e-4eec-97c4-1ef86e118ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:19.451111 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dc14e-545e-4eec-97c4-1ef86e118ae5 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:19.451664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dc14e-545e-4eec-97c4-1ef86e118ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:19.452136 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dc14e-545e-4eec-97c4-1ef86e118ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:19.453122 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449dc14e-545e-4eec-97c4-1ef86e118ae5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:19.453644 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449dc14e-545e-4eec-97c4-1ef86e118ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:19.456336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449dc14e-545e-4eec-97c4-1ef86e118ae5 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:19.467088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 165/347] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:19.468276 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449dc14e-545e-4eec-97c4-1ef86e118ae5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:19.469963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0215527c-5940-47cf-aae9-5130d1c91e4a None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:19.471255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0215527c-5940-47cf-aae9-5130d1c91e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:19.574466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 183/348] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:19.578276 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215527c-5940-47cf-aae9-5130d1c91e4a None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:19.612808 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449dc14e-545e-4eec-97c4-1ef86e118ae5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:19.652489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ce54a-5b3e-4a04-b0fb-fc027f243b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:19.653558 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ce54a-5b3e-4a04-b0fb-fc027f243b10 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:19.654172 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ce54a-5b3e-4a04-b0fb-fc027f243b10 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:19.654654 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ce54a-5b3e-4a04-b0fb-fc027f243b10 None admin] PATH_INFO: `/v3/groups/674be98aad74479f8d8548a1c1361f03` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:19.661724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:19.662100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ce54a-5b3e-4a04-b0fb-fc027f243b10 None admin] RBAC: Authorizing `identity:get_group(group_id=674be98aad74479f8d8548a1c1361f03)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:19.664109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ce54a-5b3e-4a04-b0fb-fc027f243b10 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:19.666317 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 166/349] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:19 2022] GET /identity/v3/groups/674be98aad74479f8d8548a1c1361f03 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:19.671760 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ce54a-5b3e-4a04-b0fb-fc027f243b10 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:19.693260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:19.725435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215527c-5940-47cf-aae9-5130d1c91e4a None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:19.762428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062bd6b8-703e-40d8-bd54-1808be10d049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:19.763456 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062bd6b8-703e-40d8-bd54-1808be10d049 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:19.763979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062bd6b8-703e-40d8-bd54-1808be10d049 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:19.764448 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062bd6b8-703e-40d8-bd54-1808be10d049 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:19.838186 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062bd6b8-703e-40d8-bd54-1808be10d049 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:19.840394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062bd6b8-703e-40d8-bd54-1808be10d049 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:19.859697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ce54a-5b3e-4a04-b0fb-fc027f243b10 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:19.904075 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:19.915447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1f484-dead-4f70-9310-3533213803fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:19.916064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1f484-dead-4f70-9310-3533213803fa None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:19.916347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1f484-dead-4f70-9310-3533213803fa None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:19.917382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1f484-dead-4f70-9310-3533213803fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:19.917494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1f484-dead-4f70-9310-3533213803fa None admin] RBAC: Adding query filter params (group.id=674be98aad74479f8d8548a1c1361f03, role.id=9fe465cfe8764112a4cc92f3480326b8, scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:19.917754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1f484-dead-4f70-9310-3533213803fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:19.919200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1f484-dead-4f70-9310-3533213803fa None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:19.930297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 167/350] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 24 07:32:19 2022] GET /identity/v3/role_assignments?group.id=674be98aad74479f8d8548a1c1361f03&scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74&role.id=9fe465cfe8764112a4cc92f3480326b8 => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:19.931565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9beaf-feac-4e8f-8819-51986419c7d9 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:19.931854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9beaf-feac-4e8f-8819-51986419c7d9 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:19.932135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9beaf-feac-4e8f-8819-51986419c7d9 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:19.933600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 168/351] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:19.940607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15838b11-d382-4d1f-a585-37cf3221e738 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:19.940950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15838b11-d382-4d1f-a585-37cf3221e738 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:19.941205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15838b11-d382-4d1f-a585-37cf3221e738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:19.952929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 184/352] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:19.954116 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062bd6b8-703e-40d8-bd54-1808be10d049 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:19.961288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-15838b11-d382-4d1f-a585-37cf3221e738 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:19.962513 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-15838b11-d382-4d1f-a585-37cf3221e738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:20.044863 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 169/353] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:20.045665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15838b11-d382-4d1f-a585-37cf3221e738 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:20.064717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:20.071500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062bd6b8-703e-40d8-bd54-1808be10d049 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:20.116031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b1fee-8cee-42d8-87ef-9304cb0c6012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:20.119127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b1fee-8cee-42d8-87ef-9304cb0c6012 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:20.119127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b1fee-8cee-42d8-87ef-9304cb0c6012 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:20.119127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b1fee-8cee-42d8-87ef-9304cb0c6012 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:20.128910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b1fee-8cee-42d8-87ef-9304cb0c6012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-1e4b1fee-8cee-42d8-87ef-9304cb0c6012 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:20.133968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:20.134862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:20.135658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:20.135658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:20.135658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:20.135658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:20.135658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:20.135658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:20.135658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:20.135658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:20.135658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:20.135658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:20.135658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:20.135658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:20.135658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 185/354] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:20.136305 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b1fee-8cee-42d8-87ef-9304cb0c6012 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:20.206326 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:20.259344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15838b11-d382-4d1f-a585-37cf3221e738 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:20.305114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576721d1-41e9-4cbb-bb9a-04eb42f76867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:20.305866 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576721d1-41e9-4cbb-bb9a-04eb42f76867 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:20.306148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576721d1-41e9-4cbb-bb9a-04eb42f76867 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:20.306490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576721d1-41e9-4cbb-bb9a-04eb42f76867 None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:20.310528 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576721d1-41e9-4cbb-bb9a-04eb42f76867 None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:20.312677 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576721d1-41e9-4cbb-bb9a-04eb42f76867 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:20.316673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 170/355] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:20 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:20.316673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576721d1-41e9-4cbb-bb9a-04eb42f76867 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:20.392347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b1fee-8cee-42d8-87ef-9304cb0c6012 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:20.439310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:20.466651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91d412-6128-4047-be48-ebb26d4feba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:20.468552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91d412-6128-4047-be48-ebb26d4feba2 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:20.468857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91d412-6128-4047-be48-ebb26d4feba2 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:20.469130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91d412-6128-4047-be48-ebb26d4feba2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:20.494812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576721d1-41e9-4cbb-bb9a-04eb42f76867 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:20.517304 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91d412-6128-4047-be48-ebb26d4feba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-2f91d412-6128-4047-be48-ebb26d4feba2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:20.523220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:20.524125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:20.524947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:20.524947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:20.524947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:20.524947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:20.524947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:20.524947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:20.524947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:20.524947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:20.524947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:20.524947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:20.524947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:32:20.524947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:20.524947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 186/356] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:20.525600 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91d412-6128-4047-be48-ebb26d4feba2 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:20.583182 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883c52a-35c2-48d3-acc0-c6b15fc0f8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:20.584746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c52a-35c2-48d3-acc0-c6b15fc0f8ab None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:20.585027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c52a-35c2-48d3-acc0-c6b15fc0f8ab None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:20.585297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c52a-35c2-48d3-acc0-c6b15fc0f8ab None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:20.585969 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883c52a-35c2-48d3-acc0-c6b15fc0f8ab None admin] RBAC: Adding query filter params (name=member) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:20.586273 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883c52a-35c2-48d3-acc0-c6b15fc0f8ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:20.589561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883c52a-35c2-48d3-acc0-c6b15fc0f8ab None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:20.599405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 171/357] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:20.600335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883c52a-35c2-48d3-acc0-c6b15fc0f8ab None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:20.749977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883c52a-35c2-48d3-acc0-c6b15fc0f8ab None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:20.772692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:20.800434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91d412-6128-4047-be48-ebb26d4feba2 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:20.818480 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881729cc-7b95-4a87-af29-2bbefc9ee651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:20.819099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881729cc-7b95-4a87-af29-2bbefc9ee651 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:20.819383 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881729cc-7b95-4a87-af29-2bbefc9ee651 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:20.819656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881729cc-7b95-4a87-af29-2bbefc9ee651 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:20.820321 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881729cc-7b95-4a87-af29-2bbefc9ee651 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:20.820783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881729cc-7b95-4a87-af29-2bbefc9ee651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:20.822522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881729cc-7b95-4a87-af29-2bbefc9ee651 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:20.824729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:20.833537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 172/358] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:20.834621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881729cc-7b95-4a87-af29-2bbefc9ee651 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:20.852257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4e1f6-5ffa-44a3-84ba-0c5882ad5e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:20.852951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4e1f6-5ffa-44a3-84ba-0c5882ad5e06 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:20.855109 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4e1f6-5ffa-44a3-84ba-0c5882ad5e06 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:20.855109 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4e1f6-5ffa-44a3-84ba-0c5882ad5e06 None admin] PATH_INFO: `/v3/groups/674be98aad74479f8d8548a1c1361f03` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:20.855388 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4e1f6-5ffa-44a3-84ba-0c5882ad5e06 None admin] RBAC: Authorizing `identity:get_group(group_id=674be98aad74479f8d8548a1c1361f03)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:20.856859 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4e1f6-5ffa-44a3-84ba-0c5882ad5e06 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:20.858602 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 187/359] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:20 2022] GET /identity/v3/groups/674be98aad74479f8d8548a1c1361f03 => generated 226 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:20.859687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4e1f6-5ffa-44a3-84ba-0c5882ad5e06 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:20.972305 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881729cc-7b95-4a87-af29-2bbefc9ee651 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:20.989220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4e1f6-5ffa-44a3-84ba-0c5882ad5e06 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:21.012236 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df599f-29d5-44e3-ad7e-193b0a50b367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:21.012973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df599f-29d5-44e3-ad7e-193b0a50b367 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:21.013174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df599f-29d5-44e3-ad7e-193b0a50b367 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:21.013400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df599f-29d5-44e3-ad7e-193b0a50b367 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:21.027407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df599f-29d5-44e3-ad7e-193b0a50b367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:21.032199 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a839104-57ec-4c39-a605-5adca22e087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-00df599f-29d5-44e3-ad7e-193b0a50b367 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:21.032351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:21.033219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:21.034497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:21.034497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:21.034497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:21.034497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:21.034497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:21.034497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:21.034497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:21.034497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:21.034497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:21.034497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:21.034497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:21.034497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:21.035097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a839104-57ec-4c39-a605-5adca22e087c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:21.035097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a839104-57ec-4c39-a605-5adca22e087c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:21.035097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a839104-57ec-4c39-a605-5adca22e087c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:21.035272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 173/360] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:32:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:21.035272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1201d-d0b6-45dc-9e67-0c3cd46a73b6 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:21.035272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1201d-d0b6-45dc-9e67-0c3cd46a73b6 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:21.035435 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1201d-d0b6-45dc-9e67-0c3cd46a73b6 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:21.038901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 174/361] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:21.039898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1201d-d0b6-45dc-9e67-0c3cd46a73b6 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:21.062451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:21.113676 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:21.128121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a839104-57ec-4c39-a605-5adca22e087c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:21.129651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a839104-57ec-4c39-a605-5adca22e087c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:21.169955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1201d-d0b6-45dc-9e67-0c3cd46a73b6 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:21.217918 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f225b9a-9f42-4c5b-876d-86846586d67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:21.218552 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f225b9a-9f42-4c5b-876d-86846586d67e None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:21.218822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f225b9a-9f42-4c5b-876d-86846586d67e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:21.219091 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f225b9a-9f42-4c5b-876d-86846586d67e None admin] PATH_INFO: `/v3/users/33a540304ffd43abb4f58cef11c804b3` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:21.222058 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f225b9a-9f42-4c5b-876d-86846586d67e None admin] RBAC: Authorizing `identity:get_user(user_id=33a540304ffd43abb4f58cef11c804b3)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:21.224452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f225b9a-9f42-4c5b-876d-86846586d67e None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:21.226349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 175/362] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:32:21 2022] GET /identity/v3/users/33a540304ffd43abb4f58cef11c804b3 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:21.227019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f225b9a-9f42-4c5b-876d-86846586d67e None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:21.285582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 188/363] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:21.288106 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2629a-b7a3-45ac-95f5-a0cd31f9dda2 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:21.288106 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2629a-b7a3-45ac-95f5-a0cd31f9dda2 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:21.288106 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2629a-b7a3-45ac-95f5-a0cd31f9dda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:21.307425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-27f2629a-b7a3-45ac-95f5-a0cd31f9dda2 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:21.308620 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-27f2629a-b7a3-45ac-95f5-a0cd31f9dda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:21.351059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f225b9a-9f42-4c5b-876d-86846586d67e None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:21.360556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:21.385454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 189/364] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:21.386604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2629a-b7a3-45ac-95f5-a0cd31f9dda2 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:21.386713 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:21.396312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f94273-4353-4413-8953-3884163655ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:21.397200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f94273-4353-4413-8953-3884163655ea None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:21.397679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f94273-4353-4413-8953-3884163655ea None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:21.398121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f94273-4353-4413-8953-3884163655ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:21.399000 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f94273-4353-4413-8953-3884163655ea None admin] RBAC: Adding query filter params (group.id=674be98aad74479f8d8548a1c1361f03, role.id=8183d3986255476f959797314695f92a, scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:21.399478 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f94273-4353-4413-8953-3884163655ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:21.401175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f94273-4353-4413-8953-3884163655ea None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:21.407500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 176/365] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 24 07:32:21 2022] GET /identity/v3/role_assignments?group.id=674be98aad74479f8d8548a1c1361f03&scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74&role.id=8183d3986255476f959797314695f92a => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:21.410708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f94273-4353-4413-8953-3884163655ea None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:21.551224 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2629a-b7a3-45ac-95f5-a0cd31f9dda2 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:21.629434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0ebc6-0d17-4594-9fcb-4c73a71aab4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:21.630783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0ebc6-0d17-4594-9fcb-4c73a71aab4d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:21.632001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0ebc6-0d17-4594-9fcb-4c73a71aab4d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:21.632620 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0ebc6-0d17-4594-9fcb-4c73a71aab4d None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:21.634207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0ebc6-0d17-4594-9fcb-4c73a71aab4d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:21.634527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0ebc6-0d17-4594-9fcb-4c73a71aab4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:21.635913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0ebc6-0d17-4594-9fcb-4c73a71aab4d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:21.647495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 190/366] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:21.648351 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0ebc6-0d17-4594-9fcb-4c73a71aab4d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:21.650945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f94273-4353-4413-8953-3884163655ea None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:21.693019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5d721-66c8-4270-a6fc-5adce60bb592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:21.694025 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5d721-66c8-4270-a6fc-5adce60bb592 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:21.694506 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5d721-66c8-4270-a6fc-5adce60bb592 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:21.695024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5d721-66c8-4270-a6fc-5adce60bb592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:21.696068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5d721-66c8-4270-a6fc-5adce60bb592 None admin] RBAC: Adding query filter params (role.id=27470a2fbc754a179323bb892496b013, user.id=33a540304ffd43abb4f58cef11c804b3, scope.system=all) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:21.696453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5d721-66c8-4270-a6fc-5adce60bb592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:21.698074 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5d721-66c8-4270-a6fc-5adce60bb592 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:21.709355 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:21.716370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 177/367] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Aug 24 07:32:21 2022] GET /identity/v3/role_assignments?user.id=33a540304ffd43abb4f58cef11c804b3&scope.system=all&role.id=27470a2fbc754a179323bb892496b013 => generated 510 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:21.716370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5d721-66c8-4270-a6fc-5adce60bb592 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:21.773646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:21.840260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5d721-66c8-4270-a6fc-5adce60bb592 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:21.868911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0ebc6-0d17-4594-9fcb-4c73a71aab4d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:21.883155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56744329-1bb2-4294-a2f8-e01d372d8a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:21.888670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56744329-1bb2-4294-a2f8-e01d372d8a31 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:21.888670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56744329-1bb2-4294-a2f8-e01d372d8a31 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:21.888670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56744329-1bb2-4294-a2f8-e01d372d8a31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:21.947483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e9217-bebb-4ba7-83ee-3224004d41c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:21.948139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e9217-bebb-4ba7-83ee-3224004d41c1 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:21.948424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e9217-bebb-4ba7-83ee-3224004d41c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:21.948720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e9217-bebb-4ba7-83ee-3224004d41c1 None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:21.953372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e9217-bebb-4ba7-83ee-3224004d41c1 None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:21.955025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e9217-bebb-4ba7-83ee-3224004d41c1 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:21.957155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 191/368] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:21 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:21.958019 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e9217-bebb-4ba7-83ee-3224004d41c1 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:21.974828 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56744329-1bb2-4294-a2f8-e01d372d8a31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:21.976404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56744329-1bb2-4294-a2f8-e01d372d8a31 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:22.033198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:22.108746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e9217-bebb-4ba7-83ee-3224004d41c1 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:22.116251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 178/369] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:22.117352 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56744329-1bb2-4294-a2f8-e01d372d8a31 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:22.136670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:22.187942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f0b42-1793-4244-85d4-752c0bab923e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:22.188578 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f0b42-1793-4244-85d4-752c0bab923e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:22.188906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f0b42-1793-4244-85d4-752c0bab923e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:22.189181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f0b42-1793-4244-85d4-752c0bab923e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:22.201070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f0b42-1793-4244-85d4-752c0bab923e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-043f0b42-1793-4244-85d4-752c0bab923e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:22.208827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:22.209679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:22.210474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:22.210474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:22.210474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:22.210474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:22.210474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:22.210474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:22.210474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:22.210474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:22.210474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:22.210474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:22.210474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:32:22.210474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:22.210474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 192/370] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:32:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:22.210474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f0b42-1793-4244-85d4-752c0bab923e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:22.256701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56744329-1bb2-4294-a2f8-e01d372d8a31 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:22.305287 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87956369-8d8e-4fd7-a9cd-70887df9194a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:22.306047 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87956369-8d8e-4fd7-a9cd-70887df9194a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:22.306328 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87956369-8d8e-4fd7-a9cd-70887df9194a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:22.306962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87956369-8d8e-4fd7-a9cd-70887df9194a None admin] PATH_INFO: `/v3/groups/674be98aad74479f8d8548a1c1361f03` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:22.308150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87956369-8d8e-4fd7-a9cd-70887df9194a None admin] RBAC: Authorizing `identity:get_group(group_id=674be98aad74479f8d8548a1c1361f03)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:22.309587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87956369-8d8e-4fd7-a9cd-70887df9194a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:22.316394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 179/371] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:22 2022] GET /identity/v3/groups/674be98aad74479f8d8548a1c1361f03 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:22.317470 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87956369-8d8e-4fd7-a9cd-70887df9194a None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:22.341797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:22.356217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f0b42-1793-4244-85d4-752c0bab923e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:22.414753 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c8eeb-a916-4e60-8b9d-1967f527490c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:22.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c8eeb-a916-4e60-8b9d-1967f527490c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:22.416197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c8eeb-a916-4e60-8b9d-1967f527490c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:22.416495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c8eeb-a916-4e60-8b9d-1967f527490c None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:22.417675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c8eeb-a916-4e60-8b9d-1967f527490c None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:22.420061 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c8eeb-a916-4e60-8b9d-1967f527490c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:22.421142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 193/372] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:22 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:22.426204 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c8eeb-a916-4e60-8b9d-1967f527490c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:22.465916 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87956369-8d8e-4fd7-a9cd-70887df9194a None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:22.488746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:22.522570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d0ea5-52a6-4d51-8b14-c15d544b90fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:22.522570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d0ea5-52a6-4d51-8b14-c15d544b90fc None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:22.522570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d0ea5-52a6-4d51-8b14-c15d544b90fc None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:22.523199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d0ea5-52a6-4d51-8b14-c15d544b90fc None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:22.524314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d0ea5-52a6-4d51-8b14-c15d544b90fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:22.524314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d0ea5-52a6-4d51-8b14-c15d544b90fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:22.526488 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d0ea5-52a6-4d51-8b14-c15d544b90fc None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:22.538501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 180/373] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:22.544066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01052a-b21e-41ac-9acb-5c7d1a11896f None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:22.545575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01052a-b21e-41ac-9acb-5c7d1a11896f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:22.548334 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01052a-b21e-41ac-9acb-5c7d1a11896f None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:22.549829 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 181/374] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:22.550807 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01052a-b21e-41ac-9acb-5c7d1a11896f None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:22.588561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c8eeb-a916-4e60-8b9d-1967f527490c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:22.642111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e33ea1-089d-48d5-9b86-0c71301624e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:22.642724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e33ea1-089d-48d5-9b86-0c71301624e3 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:22.644776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e33ea1-089d-48d5-9b86-0c71301624e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:22.644776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e33ea1-089d-48d5-9b86-0c71301624e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:22.644776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e33ea1-089d-48d5-9b86-0c71301624e3 None admin] RBAC: Adding query filter params (group.id=674be98aad74479f8d8548a1c1361f03, role.id=9fe465cfe8764112a4cc92f3480326b8, scope.project.id=7ca3316be8414d20996d0efd2dceee60) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:22.645036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e33ea1-089d-48d5-9b86-0c71301624e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:22.646520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e33ea1-089d-48d5-9b86-0c71301624e3 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:22.656191 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 194/375] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 24 07:32:22 2022] GET /identity/v3/role_assignments?group.id=674be98aad74479f8d8548a1c1361f03&scope.project.id=7ca3316be8414d20996d0efd2dceee60&role.id=9fe465cfe8764112a4cc92f3480326b8 => generated 260 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:22.656360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e33ea1-089d-48d5-9b86-0c71301624e3 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:22.688674 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:22.737407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01052a-b21e-41ac-9acb-5c7d1a11896f None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:22.835009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eceb183-ff43-4922-a47b-df790ecc9d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:22.835345 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceb183-ff43-4922-a47b-df790ecc9d74 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:22.835625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceb183-ff43-4922-a47b-df790ecc9d74 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:22.835956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceb183-ff43-4922-a47b-df790ecc9d74 None admin] PATH_INFO: `/v3/groups/674be98aad74479f8d8548a1c1361f03` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:22.841313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eceb183-ff43-4922-a47b-df790ecc9d74 None admin] RBAC: Authorizing `identity:get_group(group_id=674be98aad74479f8d8548a1c1361f03)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:22.841313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eceb183-ff43-4922-a47b-df790ecc9d74 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:22.841673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 182/376] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:22 2022] GET /identity/v3/groups/674be98aad74479f8d8548a1c1361f03 => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:22.842864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca03fae-561b-4fac-b229-53ca7ad0612d None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:22.843017 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca03fae-561b-4fac-b229-53ca7ad0612d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:22.843295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca03fae-561b-4fac-b229-53ca7ad0612d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:22.871439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e33ea1-089d-48d5-9b86-0c71301624e3 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:22.874208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:22.876088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-bca03fae-561b-4fac-b229-53ca7ad0612d None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:22.877695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-bca03fae-561b-4fac-b229-53ca7ad0612d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:22.913339 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bf97b-1e23-4f2d-961b-6439ffcd6b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:22.913961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bf97b-1e23-4f2d-961b-6439ffcd6b8d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:22.914243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bf97b-1e23-4f2d-961b-6439ffcd6b8d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:22.914518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bf97b-1e23-4f2d-961b-6439ffcd6b8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:22.964196 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 183/377] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:22.965574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca03fae-561b-4fac-b229-53ca7ad0612d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:23.003250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bf97b-1e23-4f2d-961b-6439ffcd6b8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:23.007116 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bf97b-1e23-4f2d-961b-6439ffcd6b8d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:23.113308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:23.170622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca03fae-561b-4fac-b229-53ca7ad0612d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:23.203446 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 195/378] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:23.213213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bf97b-1e23-4f2d-961b-6439ffcd6b8d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:23.217282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:23.245730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca9965-3095-430e-9166-fe7632fdf7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:23.246352 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca9965-3095-430e-9166-fe7632fdf7fb None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:23.246629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca9965-3095-430e-9166-fe7632fdf7fb None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:23.246901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca9965-3095-430e-9166-fe7632fdf7fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:23.247626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca9965-3095-430e-9166-fe7632fdf7fb None admin] RBAC: Adding query filter params (group.id=674be98aad74479f8d8548a1c1361f03, role.id=8183d3986255476f959797314695f92a, scope.project.id=7ca3316be8414d20996d0efd2dceee60) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:23.248101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca9965-3095-430e-9166-fe7632fdf7fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:23.249698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca9965-3095-430e-9166-fe7632fdf7fb None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:23.265850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 184/379] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 24 07:32:22 2022] GET /identity/v3/role_assignments?group.id=674be98aad74479f8d8548a1c1361f03&scope.project.id=7ca3316be8414d20996d0efd2dceee60&role.id=8183d3986255476f959797314695f92a => generated 260 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:23.327797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca9965-3095-430e-9166-fe7632fdf7fb None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:23.394966 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bf97b-1e23-4f2d-961b-6439ffcd6b8d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:23.481445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a3242-a61c-418a-b455-f442ddcfb82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:23.482003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a3242-a61c-418a-b455-f442ddcfb82b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:23.482336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a3242-a61c-418a-b455-f442ddcfb82b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:23.484690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a3242-a61c-418a-b455-f442ddcfb82b None admin] PATH_INFO: `/v3/projects/12ef7b65bdcb4d6d980c3b5faea49511` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:23.484690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a3242-a61c-418a-b455-f442ddcfb82b None admin] RBAC: Authorizing `identity:get_project(project_id=12ef7b65bdcb4d6d980c3b5faea49511)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:23.486393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a3242-a61c-418a-b455-f442ddcfb82b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:23.488283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 196/380] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:23 2022] GET /identity/v3/projects/12ef7b65bdcb4d6d980c3b5faea49511 => generated 344 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:23.500204 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a3242-a61c-418a-b455-f442ddcfb82b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:23.550542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca9965-3095-430e-9166-fe7632fdf7fb None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:23.641099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:23.675534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae69b1-1ce5-41c0-9b60-5455ca0b5005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:23.677061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae69b1-1ce5-41c0-9b60-5455ca0b5005 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:23.677561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae69b1-1ce5-41c0-9b60-5455ca0b5005 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:23.678059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae69b1-1ce5-41c0-9b60-5455ca0b5005 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:23.684684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a3242-a61c-418a-b455-f442ddcfb82b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:23.715082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:23.724766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd9f3f-5949-45a2-81ff-ade39998ac67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:23.725412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd9f3f-5949-45a2-81ff-ade39998ac67 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:23.729058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd9f3f-5949-45a2-81ff-ade39998ac67 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:23.729167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae69b1-1ce5-41c0-9b60-5455ca0b5005 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:23.729245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd9f3f-5949-45a2-81ff-ade39998ac67 None admin] PATH_INFO: `/v3/groups/1279349c50ac4b548dd17399fcff3a1a` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:23.734600 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd9f3f-5949-45a2-81ff-ade39998ac67 None admin] RBAC: Authorizing `identity:get_group(group_id=1279349c50ac4b548dd17399fcff3a1a)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:23.736144 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd9f3f-5949-45a2-81ff-ade39998ac67 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-58ae69b1-1ce5-41c0-9b60-5455ca0b5005 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:23.737800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:23.738658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:23.739999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:23.739999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:23.739999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:23.739999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:23.739999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:23.739999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:23.739999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:23.739999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:23.739999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:23.739999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:23.739999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:23.739999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:23.741206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 185/381] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:23.746573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 197/382] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:23 2022] GET /identity/v3/groups/1279349c50ac4b548dd17399fcff3a1a => generated 229 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:23.746573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd9f3f-5949-45a2-81ff-ade39998ac67 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:23.746762 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae69b1-1ce5-41c0-9b60-5455ca0b5005 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:23.870548 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd9f3f-5949-45a2-81ff-ade39998ac67 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:23.914031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34060148-e404-4fcc-a505-c64b7460833f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:23.914651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34060148-e404-4fcc-a505-c64b7460833f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:23.915329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34060148-e404-4fcc-a505-c64b7460833f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:23.915604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34060148-e404-4fcc-a505-c64b7460833f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:23.916662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34060148-e404-4fcc-a505-c64b7460833f None admin] RBAC: Adding query filter params (group.id=1279349c50ac4b548dd17399fcff3a1a, role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=12ef7b65bdcb4d6d980c3b5faea49511) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:23.918249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34060148-e404-4fcc-a505-c64b7460833f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:23.919900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34060148-e404-4fcc-a505-c64b7460833f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:23.923735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:23.932668 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 198/383] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 24 07:32:23 2022] GET /identity/v3/role_assignments?group.id=1279349c50ac4b548dd17399fcff3a1a&scope.project.id=12ef7b65bdcb4d6d980c3b5faea49511&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:23.967769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae69b1-1ce5-41c0-9b60-5455ca0b5005 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:24.069612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81160e10-571d-41bc-b9cb-4f1514ad2c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:24.070265 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81160e10-571d-41bc-b9cb-4f1514ad2c84 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:24.070553 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81160e10-571d-41bc-b9cb-4f1514ad2c84 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:24.070829 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81160e10-571d-41bc-b9cb-4f1514ad2c84 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:24.071520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81160e10-571d-41bc-b9cb-4f1514ad2c84 None admin] RBAC: Adding query filter params (name=member) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:24.071857 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81160e10-571d-41bc-b9cb-4f1514ad2c84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:24.073267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81160e10-571d-41bc-b9cb-4f1514ad2c84 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:24.093700 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 186/384] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:24.102849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34060148-e404-4fcc-a505-c64b7460833f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:24.222375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5386bab-29e8-4ac4-b94e-76c530321ae5 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:24.223129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5386bab-29e8-4ac4-b94e-76c530321ae5 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:24.223206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5386bab-29e8-4ac4-b94e-76c530321ae5 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:24.225171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 187/385] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:24.231020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767a4ff-e233-42c7-957c-a2e06d3ad8df None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:24.231309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767a4ff-e233-42c7-957c-a2e06d3ad8df None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:24.231589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767a4ff-e233-42c7-957c-a2e06d3ad8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:24.271346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34060148-e404-4fcc-a505-c64b7460833f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:24.275421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-3767a4ff-e233-42c7-957c-a2e06d3ad8df None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:24.276624 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-3767a4ff-e233-42c7-957c-a2e06d3ad8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:24.314177 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782655f-dc96-489e-b268-ed137cdfb34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:24.318195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782655f-dc96-489e-b268-ed137cdfb34f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:24.318195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782655f-dc96-489e-b268-ed137cdfb34f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:24.318195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782655f-dc96-489e-b268-ed137cdfb34f None admin] PATH_INFO: `/v3/groups/674be98aad74479f8d8548a1c1361f03` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:24.318195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782655f-dc96-489e-b268-ed137cdfb34f None admin] RBAC: Authorizing `identity:get_group(group_id=674be98aad74479f8d8548a1c1361f03)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:24.318195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782655f-dc96-489e-b268-ed137cdfb34f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:24.320980 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 199/386] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:24 2022] GET /identity/v3/groups/674be98aad74479f8d8548a1c1361f03 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:24.330419 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782655f-dc96-489e-b268-ed137cdfb34f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:24.424670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:24.433997 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:24.459464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782655f-dc96-489e-b268-ed137cdfb34f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:24.483455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 188/387] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:32:24.529488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57e2a7-80ae-4026-9520-e18037beba94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:24.529686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57e2a7-80ae-4026-9520-e18037beba94 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:24.529889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57e2a7-80ae-4026-9520-e18037beba94 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:24.530098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57e2a7-80ae-4026-9520-e18037beba94 None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:24.537236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57e2a7-80ae-4026-9520-e18037beba94 None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:24.537236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57e2a7-80ae-4026-9520-e18037beba94 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:24.537236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 200/388] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:24 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:24.541937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767a4ff-e233-42c7-957c-a2e06d3ad8df None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:24.756680 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767a4ff-e233-42c7-957c-a2e06d3ad8df None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:24.826723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c4f57-a7e7-4d08-80f4-7f383327247d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:24.827400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c4f57-a7e7-4d08-80f4-7f383327247d None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:24.827681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c4f57-a7e7-4d08-80f4-7f383327247d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:24.827957 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c4f57-a7e7-4d08-80f4-7f383327247d None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74/groups/674be98aad74479f8d8548a1c1361f03/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:24.833979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c4f57-a7e7-4d08-80f4-7f383327247d None admin] RBAC: Authorizing `identity:create_grant(project_id=0c39a23af4154a8b9ada3f37e9fa6e74, group_id=674be98aad74479f8d8548a1c1361f03, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:24.835429 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c4f57-a7e7-4d08-80f4-7f383327247d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:24.905592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57e2a7-80ae-4026-9520-e18037beba94 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:24.943251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 189/389] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Aug 24 07:32:24 2022] PUT /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74/groups/674be98aad74479f8d8548a1c1361f03/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:32:25.060750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57e2a7-80ae-4026-9520-e18037beba94 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:25.125538 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ec9b6-63cc-44ec-a7d3-a7234f97de33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:25.126208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ec9b6-63cc-44ec-a7d3-a7234f97de33 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:25.126491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ec9b6-63cc-44ec-a7d3-a7234f97de33 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:25.126766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ec9b6-63cc-44ec-a7d3-a7234f97de33 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:25.130827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:25.147601 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ec9b6-63cc-44ec-a7d3-a7234f97de33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-eb8ec9b6-63cc-44ec-a7d3-a7234f97de33 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:25.152374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:25.153282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:25.154575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:25.154575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:25.154575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:25.154575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:25.154575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:25.154575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:25.154575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:25.154575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:25.154575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:25.154575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:25.154575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:25.154575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:25.157997 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 201/390] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:32:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:32:25.165762 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c4f57-a7e7-4d08-80f4-7f383327247d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:25.297642 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:25.353421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c4f57-a7e7-4d08-80f4-7f383327247d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:25.410939 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19c5b-7bd3-4ae7-9867-9c20136d9827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:25.411991 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19c5b-7bd3-4ae7-9867-9c20136d9827 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:25.412445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19c5b-7bd3-4ae7-9867-9c20136d9827 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:25.412747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19c5b-7bd3-4ae7-9867-9c20136d9827 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:25.413430 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf19c5b-7bd3-4ae7-9867-9c20136d9827 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:25.413738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf19c5b-7bd3-4ae7-9867-9c20136d9827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:25.413915 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3340036-3da6-4942-8cba-de650f4e8fdc None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:25.414201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3340036-3da6-4942-8cba-de650f4e8fdc None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:25.414479 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3340036-3da6-4942-8cba-de650f4e8fdc None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:25.415116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf19c5b-7bd3-4ae7-9867-9c20136d9827 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:25.415814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 202/391] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:25.421093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a445e82-6a71-4dbf-a736-cb7cf8bc4cd6 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:25.421387 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a445e82-6a71-4dbf-a736-cb7cf8bc4cd6 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:25.421662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a445e82-6a71-4dbf-a736-cb7cf8bc4cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:25.425335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 190/392] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:25.436061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19c5b-7bd3-4ae7-9867-9c20136d9827 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:25.468731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-5a445e82-6a71-4dbf-a736-cb7cf8bc4cd6 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:25.470121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5a445e82-6a71-4dbf-a736-cb7cf8bc4cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:25.591806 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19c5b-7bd3-4ae7-9867-9c20136d9827 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:25.611421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:25.636939 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6808b-dd9b-412c-a19c-cac80ff44fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:25.638781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6808b-dd9b-412c-a19c-cac80ff44fd0 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:25.639229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6808b-dd9b-412c-a19c-cac80ff44fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:25.639693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6808b-dd9b-412c-a19c-cac80ff44fd0 None admin] PATH_INFO: `/v3/groups/674be98aad74479f8d8548a1c1361f03` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:25.641494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6808b-dd9b-412c-a19c-cac80ff44fd0 None admin] RBAC: Authorizing `identity:get_group(group_id=674be98aad74479f8d8548a1c1361f03)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:25.646197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6808b-dd9b-412c-a19c-cac80ff44fd0 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:25.651841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 191/393] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:25 2022] GET /identity/v3/groups/674be98aad74479f8d8548a1c1361f03 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:25.656738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6808b-dd9b-412c-a19c-cac80ff44fd0 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:25.688669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 203/394] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:25.912412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6808b-dd9b-412c-a19c-cac80ff44fd0 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:25.937361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746a0a3-2bc9-4b9c-82e9-d4af1759daaf None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:25.937651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746a0a3-2bc9-4b9c-82e9-d4af1759daaf None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:25.937929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746a0a3-2bc9-4b9c-82e9-d4af1759daaf None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:25.939275 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 204/395] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:32:25.944818 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a333581-a1b1-4a59-8e37-dd6fef365d3c None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:25.945104 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a333581-a1b1-4a59-8e37-dd6fef365d3c None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:25.945383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a333581-a1b1-4a59-8e37-dd6fef365d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:25.976426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9a333581-a1b1-4a59-8e37-dd6fef365d3c None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:25.985225 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9a333581-a1b1-4a59-8e37-dd6fef365d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:26.010249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99f221-0cb2-4c07-8f1f-6b404210bb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:26.011153 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99f221-0cb2-4c07-8f1f-6b404210bb61 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:26.011562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99f221-0cb2-4c07-8f1f-6b404210bb61 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:26.011957 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99f221-0cb2-4c07-8f1f-6b404210bb61 None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:26.013797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99f221-0cb2-4c07-8f1f-6b404210bb61 None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:26.019569 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99f221-0cb2-4c07-8f1f-6b404210bb61 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:26.023631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 192/396] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:25 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:26.029203 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99f221-0cb2-4c07-8f1f-6b404210bb61 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:26.092069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 205/397] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:26.092953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a333581-a1b1-4a59-8e37-dd6fef365d3c None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:26.118309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:26.157631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:26.209495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a333581-a1b1-4a59-8e37-dd6fef365d3c None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:26.252545 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be65d3-940d-4b6e-8cc8-b7763a9ac93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:26.254326 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be65d3-940d-4b6e-8cc8-b7763a9ac93b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:26.255389 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be65d3-940d-4b6e-8cc8-b7763a9ac93b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:26.255899 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be65d3-940d-4b6e-8cc8-b7763a9ac93b None admin] PATH_INFO: `/v3/roles/member` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:26.270165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be65d3-940d-4b6e-8cc8-b7763a9ac93b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:26.273511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99f221-0cb2-4c07-8f1f-6b404210bb61 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-e5be65d3-940d-4b6e-8cc8-b7763a9ac93b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:26.284020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:26.285078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:26.286393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:26.286393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:26.286393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:26.286393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:26.286393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:26.286393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:26.286393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:26.286393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:26.286393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:26.286393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:26.286393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:26.286393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:26.297417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 206/398] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:26.300776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be65d3-940d-4b6e-8cc8-b7763a9ac93b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:26.347146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153edc9f-6436-4132-99de-00e2ff67f99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:26.351041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153edc9f-6436-4132-99de-00e2ff67f99c None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:26.351507 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153edc9f-6436-4132-99de-00e2ff67f99c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:26.351950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153edc9f-6436-4132-99de-00e2ff67f99c None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74/groups/674be98aad74479f8d8548a1c1361f03/roles/8183d3986255476f959797314695f92a` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:26.357654 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153edc9f-6436-4132-99de-00e2ff67f99c None admin] RBAC: Authorizing `identity:create_grant(project_id=0c39a23af4154a8b9ada3f37e9fa6e74, group_id=674be98aad74479f8d8548a1c1361f03, role_id=8183d3986255476f959797314695f92a)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:26.361456 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153edc9f-6436-4132-99de-00e2ff67f99c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:26.452008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be65d3-940d-4b6e-8cc8-b7763a9ac93b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:26.478284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:26.480062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 193/399] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Aug 24 07:32:26 2022] PUT /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74/groups/674be98aad74479f8d8548a1c1361f03/roles/8183d3986255476f959797314695f92a => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:26.481427 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153edc9f-6436-4132-99de-00e2ff67f99c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:26.515595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be0eb6-fbe8-4d6d-9dcb-5e9cde2cbc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:26.517615 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be0eb6-fbe8-4d6d-9dcb-5e9cde2cbc95 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:26.519169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be0eb6-fbe8-4d6d-9dcb-5e9cde2cbc95 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:26.519848 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be0eb6-fbe8-4d6d-9dcb-5e9cde2cbc95 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:26.522819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be0eb6-fbe8-4d6d-9dcb-5e9cde2cbc95 None admin] RBAC: Adding query filter params (name=member) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:26.524073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be0eb6-fbe8-4d6d-9dcb-5e9cde2cbc95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:26.525968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be0eb6-fbe8-4d6d-9dcb-5e9cde2cbc95 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:26.546692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 207/400] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:26.553898 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be0eb6-fbe8-4d6d-9dcb-5e9cde2cbc95 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:26.685168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be0eb6-fbe8-4d6d-9dcb-5e9cde2cbc95 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:26.728144 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5969be0-d4e9-4a35-8aa0-207a52e6680f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:26.729551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5969be0-d4e9-4a35-8aa0-207a52e6680f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:26.731421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5969be0-d4e9-4a35-8aa0-207a52e6680f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:26.731889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5969be0-d4e9-4a35-8aa0-207a52e6680f None admin] PATH_INFO: `/v3/groups/674be98aad74479f8d8548a1c1361f03` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:26.733836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5969be0-d4e9-4a35-8aa0-207a52e6680f None admin] RBAC: Authorizing `identity:get_group(group_id=674be98aad74479f8d8548a1c1361f03)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:26.736030 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5969be0-d4e9-4a35-8aa0-207a52e6680f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:26.741815 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 208/401] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:26 2022] GET /identity/v3/groups/674be98aad74479f8d8548a1c1361f03 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:26.744599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5969be0-d4e9-4a35-8aa0-207a52e6680f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:26.764544 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153edc9f-6436-4132-99de-00e2ff67f99c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:26.781278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:26.816670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:26.816858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a76b99-21a2-4ba5-997b-030994e09490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:26.816858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a76b99-21a2-4ba5-997b-030994e09490 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:26.816858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a76b99-21a2-4ba5-997b-030994e09490 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:26.816858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a76b99-21a2-4ba5-997b-030994e09490 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:26.829895 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a76b99-21a2-4ba5-997b-030994e09490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-00a76b99-21a2-4ba5-997b-030994e09490 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:26.837715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:26.838749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:26.839611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:26.839611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:26.839611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:26.839611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:26.839611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:26.839611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:26.839611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:26.839611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:26.839611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:26.839611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:26.839611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:26.839611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:26.840493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 194/402] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:32:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:26.843266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc17b77-5d07-4457-91db-dc07b3568357 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:26.843266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc17b77-5d07-4457-91db-dc07b3568357 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:26.843266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc17b77-5d07-4457-91db-dc07b3568357 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:26.843266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 195/403] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:26.847158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc17b77-5d07-4457-91db-dc07b3568357 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:26.912487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5969be0-d4e9-4a35-8aa0-207a52e6680f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:26.967488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a5c28-0881-4703-89d4-3f28fc45dafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:26.968238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a5c28-0881-4703-89d4-3f28fc45dafc None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:26.968588 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a5c28-0881-4703-89d4-3f28fc45dafc None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:26.970311 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a5c28-0881-4703-89d4-3f28fc45dafc None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:26.971888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a5c28-0881-4703-89d4-3f28fc45dafc None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:26.975461 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a5c28-0881-4703-89d4-3f28fc45dafc None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:26.981194 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 209/404] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:26 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:26.984737 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78d561-a777-4f8e-a9ed-c042799517a8 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:26.986036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78d561-a777-4f8e-a9ed-c042799517a8 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:26.986390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78d561-a777-4f8e-a9ed-c042799517a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:27.013024 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-cb78d561-a777-4f8e-a9ed-c042799517a8 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:27.027769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-cb78d561-a777-4f8e-a9ed-c042799517a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:27.057824 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc17b77-5d07-4457-91db-dc07b3568357 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:27.127662 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676469e-9eac-4e06-8768-cac46dbf7560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:27.128282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676469e-9eac-4e06-8768-cac46dbf7560 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:27.128563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676469e-9eac-4e06-8768-cac46dbf7560 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:27.128858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676469e-9eac-4e06-8768-cac46dbf7560 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:27.129530 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676469e-9eac-4e06-8768-cac46dbf7560 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:27.139833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676469e-9eac-4e06-8768-cac46dbf7560 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:27.139833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676469e-9eac-4e06-8768-cac46dbf7560 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:27.140836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 210/405] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:27.141759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78d561-a777-4f8e-a9ed-c042799517a8 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:27.151321 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 196/406] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:27.168249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676469e-9eac-4e06-8768-cac46dbf7560 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:27.238451 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:27.244126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:27.360722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676469e-9eac-4e06-8768-cac46dbf7560 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:27.402524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78d561-a777-4f8e-a9ed-c042799517a8 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:27.406508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba3b19-5589-4fe7-834a-e56b91a9a87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:27.408671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba3b19-5589-4fe7-834a-e56b91a9a87a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:27.408671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba3b19-5589-4fe7-834a-e56b91a9a87a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:27.408671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba3b19-5589-4fe7-834a-e56b91a9a87a None admin] PATH_INFO: `/v3/groups/674be98aad74479f8d8548a1c1361f03` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:27.410643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba3b19-5589-4fe7-834a-e56b91a9a87a None admin] RBAC: Authorizing `identity:get_group(group_id=674be98aad74479f8d8548a1c1361f03)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:27.415162 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba3b19-5589-4fe7-834a-e56b91a9a87a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:27.415162 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 197/407] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:27 2022] GET /identity/v3/groups/674be98aad74479f8d8548a1c1361f03 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:27.415803 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71be8d-4736-4e25-9938-23e4d781b1cd None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:27.416972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71be8d-4736-4e25-9938-23e4d781b1cd None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:27.416972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71be8d-4736-4e25-9938-23e4d781b1cd None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:27.417816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 198/408] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:27.422605 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71be8d-4736-4e25-9938-23e4d781b1cd None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:27.460509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842b2db-ca71-4e71-a3f2-fa4e30d91772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:27.461245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842b2db-ca71-4e71-a3f2-fa4e30d91772 None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:27.461525 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842b2db-ca71-4e71-a3f2-fa4e30d91772 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:27.462773 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842b2db-ca71-4e71-a3f2-fa4e30d91772 None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60/groups/674be98aad74479f8d8548a1c1361f03/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:27.466907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842b2db-ca71-4e71-a3f2-fa4e30d91772 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ca3316be8414d20996d0efd2dceee60, group_id=674be98aad74479f8d8548a1c1361f03, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:27.468420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842b2db-ca71-4e71-a3f2-fa4e30d91772 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:27.542009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:27.545674 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71be8d-4736-4e25-9938-23e4d781b1cd None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:27.573068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 211/409] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Aug 24 07:32:27 2022] PUT /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60/groups/674be98aad74479f8d8548a1c1361f03/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:27.575876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2742397-6c74-4379-ab5e-b851862e726b None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:27.576176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2742397-6c74-4379-ab5e-b851862e726b None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:27.576449 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2742397-6c74-4379-ab5e-b851862e726b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:27.603521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663cd6d-ef38-4c16-b104-ec4005b8102c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:27.615391 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663cd6d-ef38-4c16-b104-ec4005b8102c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:27.615391 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663cd6d-ef38-4c16-b104-ec4005b8102c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:27.615391 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663cd6d-ef38-4c16-b104-ec4005b8102c None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:27.615391 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663cd6d-ef38-4c16-b104-ec4005b8102c None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:27.615391 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663cd6d-ef38-4c16-b104-ec4005b8102c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:27.615790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-e2742397-6c74-4379-ab5e-b851862e726b None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:27.615882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 199/410] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:27 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:27.615975 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e2742397-6c74-4379-ab5e-b851862e726b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:27.616058 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663cd6d-ef38-4c16-b104-ec4005b8102c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:27.730126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 212/411] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:27.737155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2742397-6c74-4379-ab5e-b851862e726b None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:27.746479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663cd6d-ef38-4c16-b104-ec4005b8102c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:27.783855 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ad2ee-198f-4651-8c5c-ad8a86ef0dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:27.784901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ad2ee-198f-4651-8c5c-ad8a86ef0dc5 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:27.785405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ad2ee-198f-4651-8c5c-ad8a86ef0dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:27.786450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ad2ee-198f-4651-8c5c-ad8a86ef0dc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:27.792959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:27.798943 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ad2ee-198f-4651-8c5c-ad8a86ef0dc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-525ad2ee-198f-4651-8c5c-ad8a86ef0dc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:27.804089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:27.813389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:27.814308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:27.814308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:27.814308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:27.814308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:27.814308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:27.814308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:27.814308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:27.814308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:27.814308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:27.814308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:27.814308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:32:27.814308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:27.814308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 200/412] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:32:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:27.815668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ad2ee-198f-4651-8c5c-ad8a86ef0dc5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:27.855633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:27.911888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2742397-6c74-4379-ab5e-b851862e726b None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:27.976540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ad2ee-198f-4651-8c5c-ad8a86ef0dc5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:28.002321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4fc90-1ff1-423b-8fb8-85e3c934b061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:28.003049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4fc90-1ff1-423b-8fb8-85e3c934b061 None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:28.003288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4fc90-1ff1-423b-8fb8-85e3c934b061 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:28.003570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4fc90-1ff1-423b-8fb8-85e3c934b061 None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60/groups/674be98aad74479f8d8548a1c1361f03/roles/8183d3986255476f959797314695f92a` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:28.015110 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4fc90-1ff1-423b-8fb8-85e3c934b061 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ca3316be8414d20996d0efd2dceee60, group_id=674be98aad74479f8d8548a1c1361f03, role_id=8183d3986255476f959797314695f92a)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:28.018939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4fc90-1ff1-423b-8fb8-85e3c934b061 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:28.019038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb34bb-bf09-4f9e-9937-5581241fc4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:28.019038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb34bb-bf09-4f9e-9937-5581241fc4c5 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:28.019038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb34bb-bf09-4f9e-9937-5581241fc4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:28.019038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb34bb-bf09-4f9e-9937-5581241fc4c5 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:28.020493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb34bb-bf09-4f9e-9937-5581241fc4c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:28.021114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb34bb-bf09-4f9e-9937-5581241fc4c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:28.022698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb34bb-bf09-4f9e-9937-5581241fc4c5 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:28.032850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 201/413] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:28.039682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb34bb-bf09-4f9e-9937-5581241fc4c5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:28.132800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 213/414] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Aug 24 07:32:27 2022] PUT /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60/groups/674be98aad74479f8d8548a1c1361f03/roles/8183d3986255476f959797314695f92a => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:28.133636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4fc90-1ff1-423b-8fb8-85e3c934b061 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:28.169123 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:28.191544 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb34bb-bf09-4f9e-9937-5581241fc4c5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:28.244540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bc321-1f63-47a5-9c34-f37d1bffed11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:28.246867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bc321-1f63-47a5-9c34-f37d1bffed11 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:28.246867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bc321-1f63-47a5-9c34-f37d1bffed11 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:28.247067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bc321-1f63-47a5-9c34-f37d1bffed11 None admin] PATH_INFO: `/v3/groups/1279349c50ac4b548dd17399fcff3a1a` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:28.248218 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bc321-1f63-47a5-9c34-f37d1bffed11 None admin] RBAC: Authorizing `identity:get_group(group_id=1279349c50ac4b548dd17399fcff3a1a)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:28.249842 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bc321-1f63-47a5-9c34-f37d1bffed11 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:28.251799 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 202/415] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:28 2022] GET /identity/v3/groups/1279349c50ac4b548dd17399fcff3a1a => generated 229 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:28.259192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bc321-1f63-47a5-9c34-f37d1bffed11 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:28.368318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:28.378015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4fc90-1ff1-423b-8fb8-85e3c934b061 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:28.416235 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bc321-1f63-47a5-9c34-f37d1bffed11 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:28.424217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e457ea3-3a94-41f6-a339-4b8242624b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:28.425211 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e457ea3-3a94-41f6-a339-4b8242624b40 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:28.425707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e457ea3-3a94-41f6-a339-4b8242624b40 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:28.426336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e457ea3-3a94-41f6-a339-4b8242624b40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:28.450458 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e457ea3-3a94-41f6-a339-4b8242624b40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-3e457ea3-3a94-41f6-a339-4b8242624b40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:28.459442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:28.460362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:28.461253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:28.461253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:28.461253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:28.461253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:28.461253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:28.461253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:28.461253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:28.461253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:28.461253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:28.461253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:28.461253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:28.461253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:28.463959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 214/416] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:28.469811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e457ea3-3a94-41f6-a339-4b8242624b40 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:28.525431 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978bf6d-87ab-4350-8a04-c716a05f9625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:28.526043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978bf6d-87ab-4350-8a04-c716a05f9625 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:28.526327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978bf6d-87ab-4350-8a04-c716a05f9625 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:28.526607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978bf6d-87ab-4350-8a04-c716a05f9625 None admin] PATH_INFO: `/v3/projects/12ef7b65bdcb4d6d980c3b5faea49511` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:28.533408 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978bf6d-87ab-4350-8a04-c716a05f9625 None admin] RBAC: Authorizing `identity:get_project(project_id=12ef7b65bdcb4d6d980c3b5faea49511)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:28.534892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978bf6d-87ab-4350-8a04-c716a05f9625 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:28.537813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 203/417] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:28 2022] GET /identity/v3/projects/12ef7b65bdcb4d6d980c3b5faea49511 => generated 344 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:28.545659 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978bf6d-87ab-4350-8a04-c716a05f9625 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:28.624487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e457ea3-3a94-41f6-a339-4b8242624b40 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:28.635519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:28.686200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082f2f2-3338-41dd-99a9-e2a4deebd402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:28.686790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f2f2-3338-41dd-99a9-e2a4deebd402 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:28.687070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f2f2-3338-41dd-99a9-e2a4deebd402 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:28.690827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f2f2-3338-41dd-99a9-e2a4deebd402 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:28.690827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082f2f2-3338-41dd-99a9-e2a4deebd402 None admin] RBAC: Adding query filter params (name=member) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:28.690827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082f2f2-3338-41dd-99a9-e2a4deebd402 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:28.690827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082f2f2-3338-41dd-99a9-e2a4deebd402 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:28.715291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 215/418] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:28.724388 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082f2f2-3338-41dd-99a9-e2a4deebd402 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:28.764693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:28.786675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978bf6d-87ab-4350-8a04-c716a05f9625 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:28.825599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306e054-83cf-4acf-9cd6-9df786029a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:28.827253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306e054-83cf-4acf-9cd6-9df786029a03 None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:28.828006 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306e054-83cf-4acf-9cd6-9df786029a03 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:28.828491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306e054-83cf-4acf-9cd6-9df786029a03 None admin] PATH_INFO: `/v3/projects/12ef7b65bdcb4d6d980c3b5faea49511/groups/1279349c50ac4b548dd17399fcff3a1a/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:28.834162 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306e054-83cf-4acf-9cd6-9df786029a03 None admin] RBAC: Authorizing `identity:create_grant(project_id=12ef7b65bdcb4d6d980c3b5faea49511, group_id=1279349c50ac4b548dd17399fcff3a1a, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:28.835638 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306e054-83cf-4acf-9cd6-9df786029a03 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:28.878282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082f2f2-3338-41dd-99a9-e2a4deebd402 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:28.919521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 204/419] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Aug 24 07:32:28 2022] PUT /identity/v3/projects/12ef7b65bdcb4d6d980c3b5faea49511/groups/1279349c50ac4b548dd17399fcff3a1a/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:28.922708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68520b8-c4dd-45fd-aaa1-da7740035746 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:28.922708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68520b8-c4dd-45fd-aaa1-da7740035746 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:28.922708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68520b8-c4dd-45fd-aaa1-da7740035746 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:28.922708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 205/420] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:28.930982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730acb8c-cd36-451b-8716-c64a04758de0 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:28.931706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730acb8c-cd36-451b-8716-c64a04758de0 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:28.932044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730acb8c-cd36-451b-8716-c64a04758de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:28.940720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da560b46-a7a9-4e30-b145-6d01f6d68998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:28.940720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da560b46-a7a9-4e30-b145-6d01f6d68998 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:28.940720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da560b46-a7a9-4e30-b145-6d01f6d68998 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:28.940720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da560b46-a7a9-4e30-b145-6d01f6d68998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:28.969847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-730acb8c-cd36-451b-8716-c64a04758de0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:28.973014 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-730acb8c-cd36-451b-8716-c64a04758de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:29.099804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da560b46-a7a9-4e30-b145-6d01f6d68998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:29.103407 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da560b46-a7a9-4e30-b145-6d01f6d68998 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:29.132740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:29.161284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 206/421] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:29.235142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 216/422] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:29.241235 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730acb8c-cd36-451b-8716-c64a04758de0 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:29.262022 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:29.410246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730acb8c-cd36-451b-8716-c64a04758de0 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:29.505190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2b46c-7b49-48dd-bb6f-5d573f40bfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:29.505974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2b46c-7b49-48dd-bb6f-5d573f40bfc8 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:29.508665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2b46c-7b49-48dd-bb6f-5d573f40bfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:29.508665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2b46c-7b49-48dd-bb6f-5d573f40bfc8 None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:29.509444 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2b46c-7b49-48dd-bb6f-5d573f40bfc8 None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:29.510911 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2b46c-7b49-48dd-bb6f-5d573f40bfc8 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:29.512902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 207/423] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:29 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:29.518420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da560b46-a7a9-4e30-b145-6d01f6d68998 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:29.532422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2b46c-7b49-48dd-bb6f-5d573f40bfc8 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:29.734785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:29.747162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da560b46-a7a9-4e30-b145-6d01f6d68998 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:29.766493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2b46c-7b49-48dd-bb6f-5d573f40bfc8 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:29.793014 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:29.826500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f84c0-f552-451c-a04c-9e7b5e0d51fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:29.830241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f84c0-f552-451c-a04c-9e7b5e0d51fa None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:29.830241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f84c0-f552-451c-a04c-9e7b5e0d51fa None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:29.830241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f84c0-f552-451c-a04c-9e7b5e0d51fa None admin] PATH_INFO: `/v3/groups/674be98aad74479f8d8548a1c1361f03` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:29.833163 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f84c0-f552-451c-a04c-9e7b5e0d51fa None admin] RBAC: Authorizing `identity:get_group(group_id=674be98aad74479f8d8548a1c1361f03)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:29.834181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f84c0-f552-451c-a04c-9e7b5e0d51fa None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:29.842988 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 217/424] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:29 2022] GET /identity/v3/groups/674be98aad74479f8d8548a1c1361f03 => generated 226 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:29.844700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f84c0-f552-451c-a04c-9e7b5e0d51fa None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:29.851294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb17e3a-d74a-44ca-809b-9e12d96cb0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:29.852153 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb17e3a-d74a-44ca-809b-9e12d96cb0de None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:29.852587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb17e3a-d74a-44ca-809b-9e12d96cb0de None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:29.853044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb17e3a-d74a-44ca-809b-9e12d96cb0de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:29.877616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb17e3a-d74a-44ca-809b-9e12d96cb0de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-0bb17e3a-d74a-44ca-809b-9e12d96cb0de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:29.887139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:29.888054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:29.888949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:29.888949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:29.888949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:29.888949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:29.888949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:29.888949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:29.888949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:29.888949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:29.888949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:29.888949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:29.888949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:29.888949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:29.891588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 208/425] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:32:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:29.899144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb17e3a-d74a-44ca-809b-9e12d96cb0de None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:30.091880 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb17e3a-d74a-44ca-809b-9e12d96cb0de None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:30.102139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f84c0-f552-451c-a04c-9e7b5e0d51fa None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:30.160052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abaa68-06a4-4d7f-be95-844e65e921c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:30.161393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abaa68-06a4-4d7f-be95-844e65e921c7 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:30.161773 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abaa68-06a4-4d7f-be95-844e65e921c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:30.162825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abaa68-06a4-4d7f-be95-844e65e921c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:30.163678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abaa68-06a4-4d7f-be95-844e65e921c7 None admin] RBAC: Adding query filter params (group.id=674be98aad74479f8d8548a1c1361f03, role.id=9fe465cfe8764112a4cc92f3480326b8, scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:30.164110 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abaa68-06a4-4d7f-be95-844e65e921c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:30.165979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abaa68-06a4-4d7f-be95-844e65e921c7 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:30.168392 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ad5dc-992a-4147-a014-451a6c692410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:30.169301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ad5dc-992a-4147-a014-451a6c692410 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:30.169704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ad5dc-992a-4147-a014-451a6c692410 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:30.170098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ad5dc-992a-4147-a014-451a6c692410 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:30.170883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ad5dc-992a-4147-a014-451a6c692410 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:30.172771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ad5dc-992a-4147-a014-451a6c692410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:30.175021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ad5dc-992a-4147-a014-451a6c692410 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:30.184022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 218/426] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 24 07:32:29 2022] GET /identity/v3/role_assignments?group.id=674be98aad74479f8d8548a1c1361f03&scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74&role.id=9fe465cfe8764112a4cc92f3480326b8 => generated 611 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:30.184022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e96d8-04b8-4dfe-8de5-4ba7da6211e3 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:30.184022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e96d8-04b8-4dfe-8de5-4ba7da6211e3 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:30.184022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e96d8-04b8-4dfe-8de5-4ba7da6211e3 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:30.186783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 219/427] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:30.188405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e5c20-9fb2-4a1c-85eb-68949b0f96ed None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:30.189225 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e5c20-9fb2-4a1c-85eb-68949b0f96ed None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:30.189423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e5c20-9fb2-4a1c-85eb-68949b0f96ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:30.212317 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 209/428] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:30.218631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ad5dc-992a-4147-a014-451a6c692410 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:30.232678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-706e5c20-9fb2-4a1c-85eb-68949b0f96ed None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:30.232678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-706e5c20-9fb2-4a1c-85eb-68949b0f96ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:30.239658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:30.305023 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:30.372309 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 220/429] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:30.437530 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ad5dc-992a-4147-a014-451a6c692410 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:30.475454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de00463-401c-43ea-a9d1-6531d09173e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:30.476079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de00463-401c-43ea-a9d1-6531d09173e3 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:30.476358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de00463-401c-43ea-a9d1-6531d09173e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:30.478641 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de00463-401c-43ea-a9d1-6531d09173e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:30.535838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717c053-db16-4c80-82b2-1371f4b1cd87 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:30.536474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717c053-db16-4c80-82b2-1371f4b1cd87 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:30.539626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717c053-db16-4c80-82b2-1371f4b1cd87 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:30.548672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 221/430] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:30.550626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7727ab-a09d-4186-8567-6998c85e3191 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:30.550626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7727ab-a09d-4186-8567-6998c85e3191 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:30.550626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7727ab-a09d-4186-8567-6998c85e3191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:30.562724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de00463-401c-43ea-a9d1-6531d09173e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:30.564140 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de00463-401c-43ea-a9d1-6531d09173e3 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:30.584844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:30.590402 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-db7727ab-a09d-4186-8567-6998c85e3191 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:30.594018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-db7727ab-a09d-4186-8567-6998c85e3191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:30.685571 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 210/431] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:30.691358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de00463-401c-43ea-a9d1-6531d09173e3 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:30.728605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 222/432] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:30.729908 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7727ab-a09d-4186-8567-6998c85e3191 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:30.808418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de00463-401c-43ea-a9d1-6531d09173e3 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:30.815640 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:30.849079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c04da8-9593-4156-848c-a669f3076315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:30.849974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c04da8-9593-4156-848c-a669f3076315 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:30.850284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c04da8-9593-4156-848c-a669f3076315 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:30.850556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c04da8-9593-4156-848c-a669f3076315 None admin] PATH_INFO: `/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:30.853251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c04da8-9593-4156-848c-a669f3076315 None admin] RBAC: Authorizing `identity:get_project(project_id=0c39a23af4154a8b9ada3f37e9fa6e74)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:30.857753 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c04da8-9593-4156-848c-a669f3076315 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:30.860282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 211/433] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:30 2022] GET /identity/v3/projects/0c39a23af4154a8b9ada3f37e9fa6e74 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:30.860809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7727ab-a09d-4186-8567-6998c85e3191 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:30.870014 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c04da8-9593-4156-848c-a669f3076315 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:30.873826 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:30.918132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bfa76-4939-42f1-b167-04376f3a4c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:30.919967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bfa76-4939-42f1-b167-04376f3a4c37 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:30.919967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bfa76-4939-42f1-b167-04376f3a4c37 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:30.919967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bfa76-4939-42f1-b167-04376f3a4c37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:30.936365 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bfa76-4939-42f1-b167-04376f3a4c37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-cb6bfa76-4939-42f1-b167-04376f3a4c37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:30.942145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:30.943059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:30.943876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:30.943876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:30.943876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:30.943876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:30.943876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:30.943876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:30.943876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:30.943876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:30.943876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:30.943876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:30.943876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:32:30.943876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:30.943876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 223/434] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:30.949679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bfa76-4939-42f1-b167-04376f3a4c37 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:31.017890 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c04da8-9593-4156-848c-a669f3076315 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:31.071614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3e5da-0dde-4b25-83a3-bf2ace6fe174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:31.072343 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e5da-0dde-4b25-83a3-bf2ace6fe174 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:31.072623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e5da-0dde-4b25-83a3-bf2ace6fe174 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:31.072916 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e5da-0dde-4b25-83a3-bf2ace6fe174 None admin] PATH_INFO: `/v3/groups/674be98aad74479f8d8548a1c1361f03` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:31.074229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3e5da-0dde-4b25-83a3-bf2ace6fe174 None admin] RBAC: Authorizing `identity:get_group(group_id=674be98aad74479f8d8548a1c1361f03)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:31.075789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3e5da-0dde-4b25-83a3-bf2ace6fe174 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:31.080683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 212/435] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:30 2022] GET /identity/v3/groups/674be98aad74479f8d8548a1c1361f03 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:31.081009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3e5da-0dde-4b25-83a3-bf2ace6fe174 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:31.112326 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bfa76-4939-42f1-b167-04376f3a4c37 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:31.150177 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:31.166581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6202ff-339f-4167-a393-a806ca78f17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:31.167290 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6202ff-339f-4167-a393-a806ca78f17c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:31.167646 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6202ff-339f-4167-a393-a806ca78f17c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:31.167992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6202ff-339f-4167-a393-a806ca78f17c None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:31.169655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6202ff-339f-4167-a393-a806ca78f17c None admin] RBAC: Adding query filter params (name=member) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:31.169655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6202ff-339f-4167-a393-a806ca78f17c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:31.170600 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6202ff-339f-4167-a393-a806ca78f17c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:31.181036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 224/436] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:32:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:31.182168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6202ff-339f-4167-a393-a806ca78f17c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:31.219899 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:31.222268 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3e5da-0dde-4b25-83a3-bf2ace6fe174 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:31.266923 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475f62e-4da2-414c-bcbb-52923ac9da2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:31.267551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475f62e-4da2-414c-bcbb-52923ac9da2b None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:31.267830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475f62e-4da2-414c-bcbb-52923ac9da2b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:31.268434 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475f62e-4da2-414c-bcbb-52923ac9da2b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:31.281138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475f62e-4da2-414c-bcbb-52923ac9da2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-e475f62e-4da2-414c-bcbb-52923ac9da2b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:31.287179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:31.288079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:31.288930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:31.288930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:31.288930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:31.288930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:31.288930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:31.288930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:31.288930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:31.288930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:31.288930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:31.288930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:31.288930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:31.288930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:31.291630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 213/437] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:32:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:31.292659 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475f62e-4da2-414c-bcbb-52923ac9da2b None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:31.319148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6202ff-339f-4167-a393-a806ca78f17c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:31.359225 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661cbef-bb0f-4abc-b61c-a80a48d95339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:31.359814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661cbef-bb0f-4abc-b61c-a80a48d95339 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:31.360094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661cbef-bb0f-4abc-b61c-a80a48d95339 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:31.360364 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661cbef-bb0f-4abc-b61c-a80a48d95339 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:31.361278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661cbef-bb0f-4abc-b61c-a80a48d95339 None admin] RBAC: Adding query filter params (group.id=674be98aad74479f8d8548a1c1361f03, role.id=8183d3986255476f959797314695f92a, scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:31.361519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661cbef-bb0f-4abc-b61c-a80a48d95339 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:31.362941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661cbef-bb0f-4abc-b61c-a80a48d95339 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:31.380949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 225/438] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 24 07:32:31 2022] GET /identity/v3/role_assignments?group.id=674be98aad74479f8d8548a1c1361f03&scope.project.id=0c39a23af4154a8b9ada3f37e9fa6e74&role.id=8183d3986255476f959797314695f92a => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:31.380949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661cbef-bb0f-4abc-b61c-a80a48d95339 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:31.447489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475f62e-4da2-414c-bcbb-52923ac9da2b None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:31.449823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:31.493532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30490802-7f29-453e-8440-672207e53b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:31.494368 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30490802-7f29-453e-8440-672207e53b45 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:31.494655 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30490802-7f29-453e-8440-672207e53b45 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:31.494928 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30490802-7f29-453e-8440-672207e53b45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:31.498879 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:31.507151 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661cbef-bb0f-4abc-b61c-a80a48d95339 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:31.560500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf665cc-0a09-4c10-aa0e-1bdfa004f9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:31.563406 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf665cc-0a09-4c10-aa0e-1bdfa004f9be None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:31.563406 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf665cc-0a09-4c10-aa0e-1bdfa004f9be None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:31.563406 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf665cc-0a09-4c10-aa0e-1bdfa004f9be None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:31.563406 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf665cc-0a09-4c10-aa0e-1bdfa004f9be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:31.563406 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf665cc-0a09-4c10-aa0e-1bdfa004f9be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:31.567240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf665cc-0a09-4c10-aa0e-1bdfa004f9be None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:31.576881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 226/439] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:31.577973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fd4ff-fcf3-4706-9efc-e1fbf921ac1b None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:31.578293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fd4ff-fcf3-4706-9efc-e1fbf921ac1b None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:31.578570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fd4ff-fcf3-4706-9efc-e1fbf921ac1b None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:31.579908 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 227/440] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:31.585194 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fd4ff-fcf3-4706-9efc-e1fbf921ac1b None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:31.603209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30490802-7f29-453e-8440-672207e53b45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:31.603827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30490802-7f29-453e-8440-672207e53b45 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:31.734687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fd4ff-fcf3-4706-9efc-e1fbf921ac1b None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:31.768517 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 214/441] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:31.774223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a5614-3555-4c76-a3c7-f1a21ce9be12 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:31.774591 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a5614-3555-4c76-a3c7-f1a21ce9be12 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:31.774945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a5614-3555-4c76-a3c7-f1a21ce9be12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:31.775794 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cae08-882b-41a9-b9c2-033fcf220537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:31.776686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cae08-882b-41a9-b9c2-033fcf220537 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:31.778396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cae08-882b-41a9-b9c2-033fcf220537 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:31.778396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cae08-882b-41a9-b9c2-033fcf220537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:31.781692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:31.796947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d11a5614-3555-4c76-a3c7-f1a21ce9be12 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:31.798122 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d11a5614-3555-4c76-a3c7-f1a21ce9be12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:31.807100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:31.857963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cae08-882b-41a9-b9c2-033fcf220537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:31.859435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cae08-882b-41a9-b9c2-033fcf220537 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:31.880861 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 215/442] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:31.880861 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a5614-3555-4c76-a3c7-f1a21ce9be12 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:31.991406 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 228/443] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:31.997247 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cae08-882b-41a9-b9c2-033fcf220537 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:32.005620 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a5614-3555-4c76-a3c7-f1a21ce9be12 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:32.044294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:32.052369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1d70a-4e9f-4ce2-8bda-5fa2d756dd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:32.056247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1d70a-4e9f-4ce2-8bda-5fa2d756dd0e None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:32.056247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1d70a-4e9f-4ce2-8bda-5fa2d756dd0e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:32.056247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1d70a-4e9f-4ce2-8bda-5fa2d756dd0e None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:32.056247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1d70a-4e9f-4ce2-8bda-5fa2d756dd0e None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:32.056247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1d70a-4e9f-4ce2-8bda-5fa2d756dd0e None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:32.062438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 216/444] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:31 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:32.063699 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1d70a-4e9f-4ce2-8bda-5fa2d756dd0e None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:32.072416 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:32.124231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cae08-882b-41a9-b9c2-033fcf220537 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:32.163663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d879d1-73bc-4e6b-bec2-409c6881cf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:32.164315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d879d1-73bc-4e6b-bec2-409c6881cf6d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:32.164596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d879d1-73bc-4e6b-bec2-409c6881cf6d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:32.164916 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d879d1-73bc-4e6b-bec2-409c6881cf6d None admin] PATH_INFO: `/v3/projects/7ca3316be8414d20996d0efd2dceee60` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:32.166145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d879d1-73bc-4e6b-bec2-409c6881cf6d None admin] RBAC: Authorizing `identity:get_project(project_id=7ca3316be8414d20996d0efd2dceee60)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:32.167626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d879d1-73bc-4e6b-bec2-409c6881cf6d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:32.169539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 229/445] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:31 2022] GET /identity/v3/projects/7ca3316be8414d20996d0efd2dceee60 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:32.170231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d879d1-73bc-4e6b-bec2-409c6881cf6d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:32.206916 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1d70a-4e9f-4ce2-8bda-5fa2d756dd0e None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:32.246210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d912c47-79ef-4dff-af31-f88d617f7533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:32.246887 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d912c47-79ef-4dff-af31-f88d617f7533 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:32.247211 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d912c47-79ef-4dff-af31-f88d617f7533 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:32.247492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d912c47-79ef-4dff-af31-f88d617f7533 None admin] PATH_INFO: `/v3/groups/674be98aad74479f8d8548a1c1361f03` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:32.248668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d912c47-79ef-4dff-af31-f88d617f7533 None admin] RBAC: Authorizing `identity:get_group(group_id=674be98aad74479f8d8548a1c1361f03)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:32.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d912c47-79ef-4dff-af31-f88d617f7533 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:32.251733 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 217/446] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:32 2022] GET /identity/v3/groups/674be98aad74479f8d8548a1c1361f03 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:32.252496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d912c47-79ef-4dff-af31-f88d617f7533 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:32.295520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d879d1-73bc-4e6b-bec2-409c6881cf6d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:32.321697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:32.332020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:32.332215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a04710-5e13-4c05-a024-0dadcdd101e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:32.333474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a04710-5e13-4c05-a024-0dadcdd101e3 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:32.336588 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a04710-5e13-4c05-a024-0dadcdd101e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:32.336588 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a04710-5e13-4c05-a024-0dadcdd101e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:32.345426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a04710-5e13-4c05-a024-0dadcdd101e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-29a04710-5e13-4c05-a024-0dadcdd101e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:32.351429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:32.352313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:32.353436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:32.353436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:32.353436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:32.353436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:32.353436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:32.353436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:32.353436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:32.353436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:32.353436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:32.353436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:32.353436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:32:32.353436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:32.353436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 230/447] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:32:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:32.354295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a04710-5e13-4c05-a024-0dadcdd101e3 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:32.380564 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d912c47-79ef-4dff-af31-f88d617f7533 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:32.419146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e2fc6-07ee-4c8c-aa6b-9ef1e86367e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:32.419785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e2fc6-07ee-4c8c-aa6b-9ef1e86367e5 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:32.420793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e2fc6-07ee-4c8c-aa6b-9ef1e86367e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:32.421138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e2fc6-07ee-4c8c-aa6b-9ef1e86367e5 None admin] PATH_INFO: `/v3/groups/674be98aad74479f8d8548a1c1361f03` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:32.424682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e2fc6-07ee-4c8c-aa6b-9ef1e86367e5 None admin] RBAC: Authorizing `identity:get_group(group_id=674be98aad74479f8d8548a1c1361f03)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:32.424682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e2fc6-07ee-4c8c-aa6b-9ef1e86367e5 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:32.425483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 218/448] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:32 2022] GET /identity/v3/groups/674be98aad74479f8d8548a1c1361f03 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:32.426252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e2fc6-07ee-4c8c-aa6b-9ef1e86367e5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:32.477844 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a04710-5e13-4c05-a024-0dadcdd101e3 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:32.515747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7d7b0-a929-4a40-b139-a8724ec1e8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:32.516360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7d7b0-a929-4a40-b139-a8724ec1e8e2 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:32.516752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7d7b0-a929-4a40-b139-a8724ec1e8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:32.517184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7d7b0-a929-4a40-b139-a8724ec1e8e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:32.517933 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7d7b0-a929-4a40-b139-a8724ec1e8e2 None admin] RBAC: Adding query filter params (group.id=674be98aad74479f8d8548a1c1361f03, role.id=9fe465cfe8764112a4cc92f3480326b8, scope.project.id=7ca3316be8414d20996d0efd2dceee60) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:32.518247 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7d7b0-a929-4a40-b139-a8724ec1e8e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:32.519716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7d7b0-a929-4a40-b139-a8724ec1e8e2 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:32.526854 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 231/449] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 24 07:32:32 2022] GET /identity/v3/role_assignments?group.id=674be98aad74479f8d8548a1c1361f03&scope.project.id=7ca3316be8414d20996d0efd2dceee60&role.id=9fe465cfe8764112a4cc92f3480326b8 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:32.527676 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7d7b0-a929-4a40-b139-a8724ec1e8e2 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:32.567813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e2fc6-07ee-4c8c-aa6b-9ef1e86367e5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:32.615966 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:32.622545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ecad2-f0ad-426d-abae-a5faa9eed8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:32.623175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ecad2-f0ad-426d-abae-a5faa9eed8e8 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:32.624345 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ecad2-f0ad-426d-abae-a5faa9eed8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:32.624751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ecad2-f0ad-426d-abae-a5faa9eed8e8 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:32.625531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ecad2-f0ad-426d-abae-a5faa9eed8e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:32.626078 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ecad2-f0ad-426d-abae-a5faa9eed8e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:32.627731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ecad2-f0ad-426d-abae-a5faa9eed8e8 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:32.636170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:32.643092 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 219/450] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:32.649562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ecad2-f0ad-426d-abae-a5faa9eed8e8 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:32.695704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7d7b0-a929-4a40-b139-a8724ec1e8e2 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:32.748883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2b075-d4b3-4e8f-a1a1-b38a0feeb630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:32.749637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2b075-d4b3-4e8f-a1a1-b38a0feeb630 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:32.749921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2b075-d4b3-4e8f-a1a1-b38a0feeb630 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:32.750436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2b075-d4b3-4e8f-a1a1-b38a0feeb630 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:32.751573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2b075-d4b3-4e8f-a1a1-b38a0feeb630 None admin] RBAC: Adding query filter params (group.id=674be98aad74479f8d8548a1c1361f03, role.id=8183d3986255476f959797314695f92a, scope.project.id=7ca3316be8414d20996d0efd2dceee60) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:32.751890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2b075-d4b3-4e8f-a1a1-b38a0feeb630 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:32.753409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2b075-d4b3-4e8f-a1a1-b38a0feeb630 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:32.760512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 232/451] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 24 07:32:32 2022] GET /identity/v3/role_assignments?group.id=674be98aad74479f8d8548a1c1361f03&scope.project.id=7ca3316be8414d20996d0efd2dceee60&role.id=8183d3986255476f959797314695f92a => generated 611 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:32.827471 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ecad2-f0ad-426d-abae-a5faa9eed8e8 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:32.879069 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e9273-59ea-4cfe-8bf0-714e709a03b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:32.880078 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e9273-59ea-4cfe-8bf0-714e709a03b3 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:32.880078 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e9273-59ea-4cfe-8bf0-714e709a03b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:32.880359 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e9273-59ea-4cfe-8bf0-714e709a03b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:32.976415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:32.982229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e9273-59ea-4cfe-8bf0-714e709a03b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:32.984093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e9273-59ea-4cfe-8bf0-714e709a03b3 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:33.115905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 220/452] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:32:33.117756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2b075-d4b3-4e8f-a1a1-b38a0feeb630 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:33.236708 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2b075-d4b3-4e8f-a1a1-b38a0feeb630 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:33.250867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:33.274216 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873614ab-9a55-49d3-a7e6-c355adec31c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:33.274881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873614ab-9a55-49d3-a7e6-c355adec31c0 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:33.275133 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873614ab-9a55-49d3-a7e6-c355adec31c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:33.275404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873614ab-9a55-49d3-a7e6-c355adec31c0 None admin] PATH_INFO: `/v3/projects/12ef7b65bdcb4d6d980c3b5faea49511` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:33.276670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873614ab-9a55-49d3-a7e6-c355adec31c0 None admin] RBAC: Authorizing `identity:get_project(project_id=12ef7b65bdcb4d6d980c3b5faea49511)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:33.278184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873614ab-9a55-49d3-a7e6-c355adec31c0 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:33.280621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 233/453] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:32:33 2022] GET /identity/v3/projects/12ef7b65bdcb4d6d980c3b5faea49511 => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:33.287188 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e9273-59ea-4cfe-8bf0-714e709a03b3 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:33.387644 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:33.391441 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e9273-59ea-4cfe-8bf0-714e709a03b3 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:33.425933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f1090-4019-43f2-b938-88ba2c99fcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:33.426625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f1090-4019-43f2-b938-88ba2c99fcce None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:33.426896 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f1090-4019-43f2-b938-88ba2c99fcce None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:33.427163 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f1090-4019-43f2-b938-88ba2c99fcce None admin] PATH_INFO: `/v3/groups/1279349c50ac4b548dd17399fcff3a1a` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:33.428252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f1090-4019-43f2-b938-88ba2c99fcce None admin] RBAC: Authorizing `identity:get_group(group_id=1279349c50ac4b548dd17399fcff3a1a)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:33.429704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f1090-4019-43f2-b938-88ba2c99fcce None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:33.431246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 221/454] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 24 07:32:33 2022] GET /identity/v3/groups/1279349c50ac4b548dd17399fcff3a1a => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:33.437236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873614ab-9a55-49d3-a7e6-c355adec31c0 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:33.540929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873614ab-9a55-49d3-a7e6-c355adec31c0 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:33.575273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf51161-b722-45e8-bd11-335996916844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:33.575908 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf51161-b722-45e8-bd11-335996916844 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:33.576193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf51161-b722-45e8-bd11-335996916844 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:33.576471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf51161-b722-45e8-bd11-335996916844 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:33.577240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf51161-b722-45e8-bd11-335996916844 None admin] RBAC: Adding query filter params (group.id=1279349c50ac4b548dd17399fcff3a1a, role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=12ef7b65bdcb4d6d980c3b5faea49511) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:33.577559 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf51161-b722-45e8-bd11-335996916844 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:33.579017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf51161-b722-45e8-bd11-335996916844 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:33.586090 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 234/455] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 24 07:32:33 2022] GET /identity/v3/role_assignments?group.id=1279349c50ac4b548dd17399fcff3a1a&scope.project.id=12ef7b65bdcb4d6d980c3b5faea49511&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:36.073079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f6c8d-8d36-4666-b1ac-179988580f72 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:36.073622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f6c8d-8d36-4666-b1ac-179988580f72 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:36.073622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f6c8d-8d36-4666-b1ac-179988580f72 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:36.074940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 222/456] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:36.080394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646a8e8-d4d6-45bb-b71e-70998ea6693f None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:36.080893 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646a8e8-d4d6-45bb-b71e-70998ea6693f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:36.080972 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646a8e8-d4d6-45bb-b71e-70998ea6693f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:36.141679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-e646a8e8-d4d6-45bb-b71e-70998ea6693f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:36.142998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e646a8e8-d4d6-45bb-b71e-70998ea6693f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:36.213087 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:36.286580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 235/457] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:32:36.350093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bbc63-b3d6-4466-9ba2-53e533c59e11 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:36.350341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bbc63-b3d6-4466-9ba2-53e533c59e11 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:36.350582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bbc63-b3d6-4466-9ba2-53e533c59e11 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:36.352257 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 223/458] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:36.365567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c7d1d-9afa-4f94-b3f9-95a03f409d96 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:36.365567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c7d1d-9afa-4f94-b3f9-95a03f409d96 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:36.365567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c7d1d-9afa-4f94-b3f9-95a03f409d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:36.397646 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-eb6c7d1d-9afa-4f94-b3f9-95a03f409d96 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:36.399125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-eb6c7d1d-9afa-4f94-b3f9-95a03f409d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:36.582274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 236/459] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:36.851110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bbc63-b3d6-4466-9ba2-53e533c59e11 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:37.030433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7ba68-7123-4751-b80b-07e577290b8c None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:37.030830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7ba68-7123-4751-b80b-07e577290b8c None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:37.032533 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7ba68-7123-4751-b80b-07e577290b8c None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:37.032838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 237/460] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:37.038695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bef5e-3807-4578-bb78-ba8916d511eb None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:37.039071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bef5e-3807-4578-bb78-ba8916d511eb None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:37.044665 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bef5e-3807-4578-bb78-ba8916d511eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:37.070701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bbc63-b3d6-4466-9ba2-53e533c59e11 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:37.076426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-cf1bef5e-3807-4578-bb78-ba8916d511eb None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:37.077663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-cf1bef5e-3807-4578-bb78-ba8916d511eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:37.168769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae47d5c-757d-429b-9003-0b5bd3a31121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:37.168769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae47d5c-757d-429b-9003-0b5bd3a31121 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:37.168769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae47d5c-757d-429b-9003-0b5bd3a31121 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:37.168769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae47d5c-757d-429b-9003-0b5bd3a31121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:37.200668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae47d5c-757d-429b-9003-0b5bd3a31121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:37.208666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae47d5c-757d-429b-9003-0b5bd3a31121 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:37.237831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 238/461] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:37.240675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bef5e-3807-4578-bb78-ba8916d511eb None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-0ae47d5c-757d-429b-9003-0b5bd3a31121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:37.272463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:37.273660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:37.274683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:37.274683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:37.274683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:37.280364 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 224/462] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:37.290791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae47d5c-757d-429b-9003-0b5bd3a31121 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:37.345068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:37.379259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:37.519267 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bef5e-3807-4578-bb78-ba8916d511eb None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:37.566431 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90621c-9f50-4661-a4ce-3aea77fe3efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:37.567334 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90621c-9f50-4661-a4ce-3aea77fe3efe None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:37.567619 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90621c-9f50-4661-a4ce-3aea77fe3efe None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:37.567896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90621c-9f50-4661-a4ce-3aea77fe3efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:37.588757 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90621c-9f50-4661-a4ce-3aea77fe3efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:37.595524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90621c-9f50-4661-a4ce-3aea77fe3efe None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:37.612770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae47d5c-757d-429b-9003-0b5bd3a31121 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-4a90621c-9f50-4661-a4ce-3aea77fe3efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:37.635799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:37.636809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:37.637649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:37.637649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:37.637649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:37.640838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 239/463] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:37.640838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4077c-afa2-4898-8e2a-b571daa89946 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:37.640838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4077c-afa2-4898-8e2a-b571daa89946 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:37.640838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4077c-afa2-4898-8e2a-b571daa89946 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:37.644245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 240/464] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:37.644245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4077c-afa2-4898-8e2a-b571daa89946 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:37.675424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3c10a-c749-41bd-93ff-3e48d4d37d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:37.675947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3c10a-c749-41bd-93ff-3e48d4d37d64 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:37.676276 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3c10a-c749-41bd-93ff-3e48d4d37d64 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:37.677084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3c10a-c749-41bd-93ff-3e48d4d37d64 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:37.679765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3c10a-c749-41bd-93ff-3e48d4d37d64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:37.680087 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3c10a-c749-41bd-93ff-3e48d4d37d64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:37.681583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3c10a-c749-41bd-93ff-3e48d4d37d64 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:37.695864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 225/465] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:37.696540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3c10a-c749-41bd-93ff-3e48d4d37d64 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:37.779412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:37.788956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:37.796122 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4077c-afa2-4898-8e2a-b571daa89946 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:37.814379 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3c10a-c749-41bd-93ff-3e48d4d37d64 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:37.831635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27aadd-cb88-4a6a-a5c5-8b7b26de5b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:37.832274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27aadd-cb88-4a6a-a5c5-8b7b26de5b10 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:37.833121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27aadd-cb88-4a6a-a5c5-8b7b26de5b10 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:37.833489 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27aadd-cb88-4a6a-a5c5-8b7b26de5b10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:37.843806 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27aadd-cb88-4a6a-a5c5-8b7b26de5b10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:37.845303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27aadd-cb88-4a6a-a5c5-8b7b26de5b10 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:37.851760 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187af4b-50e0-4edb-9386-7c3249d45691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:37.852374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187af4b-50e0-4edb-9386-7c3249d45691 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:37.852665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187af4b-50e0-4edb-9386-7c3249d45691 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:37.852987 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187af4b-50e0-4edb-9386-7c3249d45691 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:37.853625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b187af4b-50e0-4edb-9386-7c3249d45691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:37.853940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b187af4b-50e0-4edb-9386-7c3249d45691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:37.855314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b187af4b-50e0-4edb-9386-7c3249d45691 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-cc27aadd-cb88-4a6a-a5c5-8b7b26de5b10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:37.862042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:37.862959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:37.864352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:37.864352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:37.864352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:37.864352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 241/466] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:37.865039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811b022-5f5c-4fb5-8336-1ae126527e42 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:37.865147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811b022-5f5c-4fb5-8336-1ae126527e42 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:37.865343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811b022-5f5c-4fb5-8336-1ae126527e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:37.872294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 226/467] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:37.873494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187af4b-50e0-4edb-9386-7c3249d45691 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:37.887496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-6811b022-5f5c-4fb5-8336-1ae126527e42 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:37.888723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6811b022-5f5c-4fb5-8336-1ae126527e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:37.962565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 242/468] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:37.963406 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811b022-5f5c-4fb5-8336-1ae126527e42 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:37.990948 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187af4b-50e0-4edb-9386-7c3249d45691 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:38.027705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a972810-9846-4c75-9fb9-667c7e48e986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:38.028439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a972810-9846-4c75-9fb9-667c7e48e986 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:38.028747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a972810-9846-4c75-9fb9-667c7e48e986 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:38.029021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a972810-9846-4c75-9fb9-667c7e48e986 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:38.029706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a972810-9846-4c75-9fb9-667c7e48e986 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:38.031080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a972810-9846-4c75-9fb9-667c7e48e986 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:38.032687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:38.041834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:38.066718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811b022-5f5c-4fb5-8336-1ae126527e42 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:38.101651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57c085-a4ff-4fb6-abc6-96cd47e553b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:38.102253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57c085-a4ff-4fb6-abc6-96cd47e553b8 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:38.102729 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57c085-a4ff-4fb6-abc6-96cd47e553b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:38.103015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57c085-a4ff-4fb6-abc6-96cd47e553b8 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:38.103649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57c085-a4ff-4fb6-abc6-96cd47e553b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:38.103982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57c085-a4ff-4fb6-abc6-96cd47e553b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:38.105387 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57c085-a4ff-4fb6-abc6-96cd47e553b8 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:38.122698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 243/469] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:38.123461 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57c085-a4ff-4fb6-abc6-96cd47e553b8 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:38.131203 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 227/470] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:32:37 2022] POST /identity/v3/users => generated 246 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:32:38.132421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e02694-47e6-4d85-b8bd-7440256d5b64 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:38.132728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e02694-47e6-4d85-b8bd-7440256d5b64 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:38.133015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e02694-47e6-4d85-b8bd-7440256d5b64 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:38.138625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 228/471] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:38.139580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e02694-47e6-4d85-b8bd-7440256d5b64 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:38.266279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e02694-47e6-4d85-b8bd-7440256d5b64 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:38.275381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57c085-a4ff-4fb6-abc6-96cd47e553b8 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:38.303486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1b4bc-d25b-490c-a572-9eae442ac1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:38.304102 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1b4bc-d25b-490c-a572-9eae442ac1e0 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:38.304384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1b4bc-d25b-490c-a572-9eae442ac1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:38.304744 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1b4bc-d25b-490c-a572-9eae442ac1e0 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:38.305344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1b4bc-d25b-490c-a572-9eae442ac1e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:38.306771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1b4bc-d25b-490c-a572-9eae442ac1e0 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:38.308675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:38.316584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af2a18-9f79-487f-be88-5bcba6a1c8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:38.317446 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af2a18-9f79-487f-be88-5bcba6a1c8c9 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:38.323102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af2a18-9f79-487f-be88-5bcba6a1c8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:38.323102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af2a18-9f79-487f-be88-5bcba6a1c8c9 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:38.323102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af2a18-9f79-487f-be88-5bcba6a1c8c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:38.323102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af2a18-9f79-487f-be88-5bcba6a1c8c9 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:38.412998 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 229/472] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:32:38 2022] POST /identity/v3/users => generated 248 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:32:38.414407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecce463-5849-4823-b6e5-af8522c5b89c None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:38.414695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecce463-5849-4823-b6e5-af8522c5b89c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:38.414967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecce463-5849-4823-b6e5-af8522c5b89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:38.435635 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-1ecce463-5849-4823-b6e5-af8522c5b89c None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:38.436878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-1ecce463-5849-4823-b6e5-af8522c5b89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:38.445348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 244/473] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:32:38 2022] POST /identity/v3/users => generated 248 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:32:38.446092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af2a18-9f79-487f-be88-5bcba6a1c8c9 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:38.554277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:38.599618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 230/474] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:38.643833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af2a18-9f79-487f-be88-5bcba6a1c8c9 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:38.684801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f09315-b599-4784-8eff-9abd212c274e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:38.685438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f09315-b599-4784-8eff-9abd212c274e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:38.685720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f09315-b599-4784-8eff-9abd212c274e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:38.686001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f09315-b599-4784-8eff-9abd212c274e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:38.697496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f09315-b599-4784-8eff-9abd212c274e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:38.698964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f09315-b599-4784-8eff-9abd212c274e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-07f09315-b599-4784-8eff-9abd212c274e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:38.716025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:38.716990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:38.718443 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:38.718443 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:38.718443 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:38.718443 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 245/475] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:38.724237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecce463-5849-4823-b6e5-af8522c5b89c None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:38.864731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecce463-5849-4823-b6e5-af8522c5b89c None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:38.914824 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5f64a-8307-4c30-9548-b124662c2992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:38.915449 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5f64a-8307-4c30-9548-b124662c2992 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:38.915729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5f64a-8307-4c30-9548-b124662c2992 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:38.916003 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5f64a-8307-4c30-9548-b124662c2992 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:38.916672 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5f64a-8307-4c30-9548-b124662c2992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:38.916986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5f64a-8307-4c30-9548-b124662c2992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:38.918371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5f64a-8307-4c30-9548-b124662c2992 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:38.935504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 231/476] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:38.940904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f09315-b599-4784-8eff-9abd212c274e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:38.958915 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5f64a-8307-4c30-9548-b124662c2992 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:39.014675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:39.049910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:39.130330 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f09315-b599-4784-8eff-9abd212c274e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:39.171789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f911e1-3987-4eb8-8fe4-9d50eb294d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:39.173012 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f911e1-3987-4eb8-8fe4-9d50eb294d76 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:39.173546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f911e1-3987-4eb8-8fe4-9d50eb294d76 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:39.174011 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f911e1-3987-4eb8-8fe4-9d50eb294d76 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:39.177412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f911e1-3987-4eb8-8fe4-9d50eb294d76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:39.179090 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f911e1-3987-4eb8-8fe4-9d50eb294d76 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:39.183400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5f64a-8307-4c30-9548-b124662c2992 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:39.232955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3e413-fd5d-4eb0-9982-0c272b34befa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:39.233952 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3e413-fd5d-4eb0-9982-0c272b34befa None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:39.234428 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3e413-fd5d-4eb0-9982-0c272b34befa None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:39.234916 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3e413-fd5d-4eb0-9982-0c272b34befa None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:39.235825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3e413-fd5d-4eb0-9982-0c272b34befa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:39.237459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3e413-fd5d-4eb0-9982-0c272b34befa None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:39.280837 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 246/477] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:32:38 2022] POST /identity/v3/users => generated 249 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:32:39.320953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.sql.core [None req-40a3e413-fd5d-4eb0-9982-0c272b34befa None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 07:32:39.320953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 07:32:39.320953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [parameters: {'id': '0fac34c839904f91a3036bebc4c0efd6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 07:32:39.320953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88250) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-40a3e413-fd5d-4eb0-9982-0c272b34befa None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application result.read() Aug 24 07:32:39.346520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:32:39.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application yield resource Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise self.value Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application session.commit() Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 07:32:39.350085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self.session.flush() Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application compat.raise_( Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 07:32:39.351580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:32:39.355144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application result.read() Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:32:39.356856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [parameters: {'id': '0fac34c839904f91a3036bebc4c0efd6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:39.358516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 07:32:39.361816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:39.368673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 232/478] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Aug 24 07:32:38 2022] POST /identity/v3/roles => generated 144 bytes in 406 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:39.370280 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f911e1-3987-4eb8-8fe4-9d50eb294d76 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:39.560728 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:39.572319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f911e1-3987-4eb8-8fe4-9d50eb294d76 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:39.619107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70005c-2709-4618-b94b-312bfd687252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:39.619799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70005c-2709-4618-b94b-312bfd687252 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:39.620081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70005c-2709-4618-b94b-312bfd687252 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:39.620356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70005c-2709-4618-b94b-312bfd687252 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:39.632120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70005c-2709-4618-b94b-312bfd687252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-ca70005c-2709-4618-b94b-312bfd687252 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:39.637208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:39.638069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:39.639001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:39.639001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:39.639001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:39.639001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:39.639001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:39.639001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:39.639001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:39.639001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:39.639001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:39.639001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:39.639001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:32:39.639001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:39.639001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 247/479] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:32:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:39.646545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3e413-fd5d-4eb0-9982-0c272b34befa None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:39.767935 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3e413-fd5d-4eb0-9982-0c272b34befa None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:39.770891 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:39.808462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854679ef-d12f-4d60-94eb-0ca377a19191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:39.809106 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854679ef-d12f-4d60-94eb-0ca377a19191 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:39.809389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854679ef-d12f-4d60-94eb-0ca377a19191 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:39.809664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854679ef-d12f-4d60-94eb-0ca377a19191 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:39.810333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854679ef-d12f-4d60-94eb-0ca377a19191 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:39.810639 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854679ef-d12f-4d60-94eb-0ca377a19191 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:39.812143 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854679ef-d12f-4d60-94eb-0ca377a19191 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:39.825675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 233/480] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:40.435657 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25411343-e65c-4fb7-8563-12c6a65c2173 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:40.436172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25411343-e65c-4fb7-8563-12c6a65c2173 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:40.436246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25411343-e65c-4fb7-8563-12c6a65c2173 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:40.437848 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 248/481] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:40.449114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f7293-7782-4dd2-967f-3dc48a5210e1 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:40.449553 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f7293-7782-4dd2-967f-3dc48a5210e1 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:40.449746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f7293-7782-4dd2-967f-3dc48a5210e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:40.474465 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-b75f7293-7782-4dd2-967f-3dc48a5210e1 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:40.475666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b75f7293-7782-4dd2-967f-3dc48a5210e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:40.558548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 234/482] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:40.894155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25411343-e65c-4fb7-8563-12c6a65c2173 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:40.904673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0e7a9-0670-40ff-958c-b902f5f8e2a8 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:40.906668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0e7a9-0670-40ff-958c-b902f5f8e2a8 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:40.906949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0e7a9-0670-40ff-958c-b902f5f8e2a8 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:40.908799 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 235/483] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:40 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:40.919472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90155e1d-7c36-4c0c-aa94-ebaf614dd458 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:40.919759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90155e1d-7c36-4c0c-aa94-ebaf614dd458 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:40.920037 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90155e1d-7c36-4c0c-aa94-ebaf614dd458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:40.949670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-90155e1d-7c36-4c0c-aa94-ebaf614dd458 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:40.950864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-90155e1d-7c36-4c0c-aa94-ebaf614dd458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:41.030366 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 236/484] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:32:41.031642 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9754e19-2df8-4d02-8659-ce8e1f999ac5 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:41.031930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9754e19-2df8-4d02-8659-ce8e1f999ac5 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:41.032205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9754e19-2df8-4d02-8659-ce8e1f999ac5 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:41.033729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 237/485] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:41.039902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5355593-4765-46fa-86c8-099bed9e495f None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:41.040186 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5355593-4765-46fa-86c8-099bed9e495f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:41.040461 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5355593-4765-46fa-86c8-099bed9e495f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:41.061699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25411343-e65c-4fb7-8563-12c6a65c2173 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:41.068860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-a5355593-4765-46fa-86c8-099bed9e495f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:41.070051 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-a5355593-4765-46fa-86c8-099bed9e495f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:41.104990 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:41.124219 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe38a19-22ab-4821-aee8-c2d699aeed84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:41.124952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe38a19-22ab-4821-aee8-c2d699aeed84 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:41.125267 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe38a19-22ab-4821-aee8-c2d699aeed84 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:41.125537 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe38a19-22ab-4821-aee8-c2d699aeed84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:41.143215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe38a19-22ab-4821-aee8-c2d699aeed84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-afe38a19-22ab-4821-aee8-c2d699aeed84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:41.151663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:41.152529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:41.153349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:41.153349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:41.153349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:41.153349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:41.153349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:41.153349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:41.153349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:41.153349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:41.153349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:41.153349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:41.153349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:41.153349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:41.157674 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 249/486] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:41.164539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe38a19-22ab-4821-aee8-c2d699aeed84 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:41.189522 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:41.191080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 238/487] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:41.297890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe38a19-22ab-4821-aee8-c2d699aeed84 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:41.350234 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e132809-6804-4017-8a4e-92b2440fa3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:41.350853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e132809-6804-4017-8a4e-92b2440fa3d7 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:41.351135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e132809-6804-4017-8a4e-92b2440fa3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:41.351411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e132809-6804-4017-8a4e-92b2440fa3d7 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:41.352078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e132809-6804-4017-8a4e-92b2440fa3d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:41.352381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e132809-6804-4017-8a4e-92b2440fa3d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:41.353868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e132809-6804-4017-8a4e-92b2440fa3d7 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:41.367943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 250/488] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:41.373838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e132809-6804-4017-8a4e-92b2440fa3d7 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:41.374051 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5355593-4765-46fa-86c8-099bed9e495f None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:41.483156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5355593-4765-46fa-86c8-099bed9e495f None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:41.501525 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:41.523648 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc0732-1873-41d4-ae11-480ed37b70f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:41.524383 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc0732-1873-41d4-ae11-480ed37b70f1 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:41.527882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc0732-1873-41d4-ae11-480ed37b70f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:41.527882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc0732-1873-41d4-ae11-480ed37b70f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:41.552288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc0732-1873-41d4-ae11-480ed37b70f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:41.557912 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e132809-6804-4017-8a4e-92b2440fa3d7 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-c1bc0732-1873-41d4-ae11-480ed37b70f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:41.563297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:41.565443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:41.567574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:41.567574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:41.567574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:41.567574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:41.567574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:41.567574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:41.567574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:41.567574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:41.567574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:41.567574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:41.567574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:41.567574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:41.567574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 239/489] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:41.573186 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc0732-1873-41d4-ae11-480ed37b70f1 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:41.590746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:41.635659 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819d0b3-7a01-46d6-9610-50a3904fb8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:41.636290 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819d0b3-7a01-46d6-9610-50a3904fb8d5 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:41.636572 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819d0b3-7a01-46d6-9610-50a3904fb8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:41.636887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819d0b3-7a01-46d6-9610-50a3904fb8d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:41.649091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819d0b3-7a01-46d6-9610-50a3904fb8d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:41.650668 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819d0b3-7a01-46d6-9610-50a3904fb8d5 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-a819d0b3-7a01-46d6-9610-50a3904fb8d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:41.666821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:41.668990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:41.670985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:41.670985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:41.670985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:41.670985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 251/490] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:41.670985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819d0b3-7a01-46d6-9610-50a3904fb8d5 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:41.693422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc0732-1873-41d4-ae11-480ed37b70f1 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:41.746064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b06632-cf1a-42b8-be93-e5b3e68a9016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:41.746592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b06632-cf1a-42b8-be93-e5b3e68a9016 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:41.746871 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b06632-cf1a-42b8-be93-e5b3e68a9016 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:41.747144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b06632-cf1a-42b8-be93-e5b3e68a9016 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:41.747805 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b06632-cf1a-42b8-be93-e5b3e68a9016 None admin] RBAC: Adding query filter params (name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:41.748107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b06632-cf1a-42b8-be93-e5b3e68a9016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:41.749509 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b06632-cf1a-42b8-be93-e5b3e68a9016 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:41.758794 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 240/491] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:41.759619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b06632-cf1a-42b8-be93-e5b3e68a9016 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:41.790325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819d0b3-7a01-46d6-9610-50a3904fb8d5 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:41.815576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:41.824290 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a43b19-f00d-40fd-b616-a9d8b5281789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:41.825005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a43b19-f00d-40fd-b616-a9d8b5281789 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:41.825285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a43b19-f00d-40fd-b616-a9d8b5281789 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:41.825555 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a43b19-f00d-40fd-b616-a9d8b5281789 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:41.836576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a43b19-f00d-40fd-b616-a9d8b5281789 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-16a43b19-f00d-40fd-b616-a9d8b5281789 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:41.841629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:41.842695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:41.849426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:41.849426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:41.849426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:41.849426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:41.849426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:41.849426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:41.849426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:41.849426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:41.849426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:41.849426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:41.849426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:41.849426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:41.849426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 252/492] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:41.849426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a43b19-f00d-40fd-b616-a9d8b5281789 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:41.853397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:41.875610 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b06632-cf1a-42b8-be93-e5b3e68a9016 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:41.912373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3eee42-b6e0-4132-9969-07759fdce67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:41.913017 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eee42-b6e0-4132-9969-07759fdce67b None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:41.913302 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eee42-b6e0-4132-9969-07759fdce67b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:41.913577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eee42-b6e0-4132-9969-07759fdce67b None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:41.914213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3eee42-b6e0-4132-9969-07759fdce67b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:41.914518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3eee42-b6e0-4132-9969-07759fdce67b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:41.915882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3eee42-b6e0-4132-9969-07759fdce67b None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:41.927544 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 241/493] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:41.928776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cc422-5424-44c7-9c3f-ee9b19f8929f None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:41.929064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cc422-5424-44c7-9c3f-ee9b19f8929f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:41.929342 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cc422-5424-44c7-9c3f-ee9b19f8929f None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:41.930689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 242/494] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:41.935221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cc422-5424-44c7-9c3f-ee9b19f8929f None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:41.968150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a43b19-f00d-40fd-b616-a9d8b5281789 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:42.004969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f240fd9-3c73-4f0d-bb5e-73f68711fe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:42.006187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f240fd9-3c73-4f0d-bb5e-73f68711fe7b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:42.006756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f240fd9-3c73-4f0d-bb5e-73f68711fe7b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:42.007200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f240fd9-3c73-4f0d-bb5e-73f68711fe7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:42.018973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f240fd9-3c73-4f0d-bb5e-73f68711fe7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:42.022597 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f240fd9-3c73-4f0d-bb5e-73f68711fe7b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-9f240fd9-3c73-4f0d-bb5e-73f68711fe7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:42.038124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:42.039032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:42.039897 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:42.039897 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:42.039897 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:42.040068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cc422-5424-44c7-9c3f-ee9b19f8929f None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:42.041796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 253/495] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:42.042710 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f240fd9-3c73-4f0d-bb5e-73f68711fe7b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:42.079373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583726f3-6d9b-4043-9cc5-8667f6a07188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:42.080158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583726f3-6d9b-4043-9cc5-8667f6a07188 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:42.080438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583726f3-6d9b-4043-9cc5-8667f6a07188 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:42.080741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583726f3-6d9b-4043-9cc5-8667f6a07188 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:42.081399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583726f3-6d9b-4043-9cc5-8667f6a07188 None admin] RBAC: Adding query filter params (name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:42.081703 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583726f3-6d9b-4043-9cc5-8667f6a07188 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:42.083635 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583726f3-6d9b-4043-9cc5-8667f6a07188 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:42.085359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:42.089550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:42.093341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 243/496] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:42.099296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d993e-3c37-4433-a66b-7f3b1c2e44ea None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:42.099911 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d993e-3c37-4433-a66b-7f3b1c2e44ea None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:42.099911 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d993e-3c37-4433-a66b-7f3b1c2e44ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:42.117586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-8b1d993e-3c37-4433-a66b-7f3b1c2e44ea None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:42.118768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-8b1d993e-3c37-4433-a66b-7f3b1c2e44ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:42.151762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f240fd9-3c73-4f0d-bb5e-73f68711fe7b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:42.187081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c3ce5-719d-4214-95b1-c5e507c3ef2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:42.187980 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c3ce5-719d-4214-95b1-c5e507c3ef2f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:42.188285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c3ce5-719d-4214-95b1-c5e507c3ef2f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:42.188524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c3ce5-719d-4214-95b1-c5e507c3ef2f None admin] PATH_INFO: `/v3/users/nova` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:42.200667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 244/497] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:42.200667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d993e-3c37-4433-a66b-7f3b1c2e44ea None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:42.206346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c3ce5-719d-4214-95b1-c5e507c3ef2f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:42.208750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c3ce5-719d-4214-95b1-c5e507c3ef2f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-135c3ce5-719d-4214-95b1-c5e507c3ef2f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:32:42.224482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:42.225436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:42.226310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:42.226310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:42.226310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:32:42.226310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:32:42.226310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:32:42.226310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:32:42.226310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:32:42.226310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:32:42.226310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:32:42.226310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:42.226815 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 254/498] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 24 07:32:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:42.230979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c3ce5-719d-4214-95b1-c5e507c3ef2f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:42.312318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d993e-3c37-4433-a66b-7f3b1c2e44ea None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:42.344039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:42.350578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b0536-d77c-4bcf-8332-b1f283eb770d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:42.353658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b0536-d77c-4bcf-8332-b1f283eb770d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:42.353658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b0536-d77c-4bcf-8332-b1f283eb770d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:42.353658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b0536-d77c-4bcf-8332-b1f283eb770d None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:42.353903 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:42.353986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b0536-d77c-4bcf-8332-b1f283eb770d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:42.353986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b0536-d77c-4bcf-8332-b1f283eb770d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:42.355514 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b0536-d77c-4bcf-8332-b1f283eb770d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:42.357261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c3ce5-719d-4214-95b1-c5e507c3ef2f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:42.367981 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 245/499] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:42.368857 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b0536-d77c-4bcf-8332-b1f283eb770d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:42.403694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d83a31-9830-41e0-89ea-9036ed049242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:42.404585 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d83a31-9830-41e0-89ea-9036ed049242 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:42.404906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d83a31-9830-41e0-89ea-9036ed049242 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:42.405180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d83a31-9830-41e0-89ea-9036ed049242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:42.420305 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d83a31-9830-41e0-89ea-9036ed049242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:42.421807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d83a31-9830-41e0-89ea-9036ed049242 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-45d83a31-9830-41e0-89ea-9036ed049242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:42.437887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:42.438820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:42.439650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:42.439650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:42.439650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:42.439650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 255/500] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:42.439857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d83a31-9830-41e0-89ea-9036ed049242 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:42.482306 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b0536-d77c-4bcf-8332-b1f283eb770d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:42.518386 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936bd06-f003-4ca3-8ca8-2ea1f489f0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:42.519034 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936bd06-f003-4ca3-8ca8-2ea1f489f0c0 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:42.519246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936bd06-f003-4ca3-8ca8-2ea1f489f0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:42.519517 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936bd06-f003-4ca3-8ca8-2ea1f489f0c0 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:42.520333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936bd06-f003-4ca3-8ca8-2ea1f489f0c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:42.520636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936bd06-f003-4ca3-8ca8-2ea1f489f0c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:42.522073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936bd06-f003-4ca3-8ca8-2ea1f489f0c0 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:42.545569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d83a31-9830-41e0-89ea-9036ed049242 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:42.558627 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 246/501] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 24 07:32:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:42.559848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d1357-2274-4314-8bea-5cb52c917ee8 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:42.560133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d1357-2274-4314-8bea-5cb52c917ee8 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:42.560407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d1357-2274-4314-8bea-5cb52c917ee8 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:42.561761 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 247/502] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:42.562296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d1357-2274-4314-8bea-5cb52c917ee8 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:42.584134 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699f229-7b5f-4a57-b6dc-b32c9d2505ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:42.584719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f229-7b5f-4a57-b6dc-b32c9d2505ab None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:42.585001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f229-7b5f-4a57-b6dc-b32c9d2505ab None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:42.585274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f229-7b5f-4a57-b6dc-b32c9d2505ab None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:42.592572 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699f229-7b5f-4a57-b6dc-b32c9d2505ab None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:42.594059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699f229-7b5f-4a57-b6dc-b32c9d2505ab None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-5699f229-7b5f-4a57-b6dc-b32c9d2505ab None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:32:42.606259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:42.607360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:42.608268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:42.608268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:42.608268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:32:42.608268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:32:42.608268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:32:42.608268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:32:42.608268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:32:42.608268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:32:42.608268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:32:42.608268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:42.608268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 256/503] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:42.608268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699f229-7b5f-4a57-b6dc-b32c9d2505ab None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:42.618872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:42.620397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:42.670048 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d1357-2274-4314-8bea-5cb52c917ee8 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:42.703193 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c0ef9-1425-4d7c-a316-583581fe37e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:42.703792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c0ef9-1425-4d7c-a316-583581fe37e6 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:42.704073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c0ef9-1425-4d7c-a316-583581fe37e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:42.704201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699f229-7b5f-4a57-b6dc-b32c9d2505ab None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:42.704348 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c0ef9-1425-4d7c-a316-583581fe37e6 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:42.705017 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c0ef9-1425-4d7c-a316-583581fe37e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:42.705322 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c0ef9-1425-4d7c-a316-583581fe37e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:42.706697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c0ef9-1425-4d7c-a316-583581fe37e6 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:42.718244 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 248/504] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:42.718244 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c0ef9-1425-4d7c-a316-583581fe37e6 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:42.737734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289494bd-a641-40a4-9ce0-632d1d435184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:42.738356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289494bd-a641-40a4-9ce0-632d1d435184 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:42.738685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289494bd-a641-40a4-9ce0-632d1d435184 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:42.738962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289494bd-a641-40a4-9ce0-632d1d435184 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:42.748927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289494bd-a641-40a4-9ce0-632d1d435184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-289494bd-a641-40a4-9ce0-632d1d435184 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:42.753976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:42.755547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:42.756347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:42.756347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:42.756347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:42.756347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:42.756347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:42.756347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:42.756347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:42.756347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:42.756347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:42.756347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:42.756347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:42.756347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:42.756347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 257/505] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:42.757594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbe04e-8d42-4172-a26b-9ac38e8daca4 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:42.757594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbe04e-8d42-4172-a26b-9ac38e8daca4 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:42.757836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbe04e-8d42-4172-a26b-9ac38e8daca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:42.775531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-34fbe04e-8d42-4172-a26b-9ac38e8daca4 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:42.776797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-34fbe04e-8d42-4172-a26b-9ac38e8daca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:42.821330 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c0ef9-1425-4d7c-a316-583581fe37e6 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:42.830455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:42.844363 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:42.844363 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 258/506] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:42.844363 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbe04e-8d42-4172-a26b-9ac38e8daca4 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:42.857157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69537483-2a9d-41c4-91de-301abc9a9cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:42.857765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69537483-2a9d-41c4-91de-301abc9a9cea None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:42.858048 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69537483-2a9d-41c4-91de-301abc9a9cea None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:42.858332 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69537483-2a9d-41c4-91de-301abc9a9cea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:42.929801 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69537483-2a9d-41c4-91de-301abc9a9cea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:42.931809 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69537483-2a9d-41c4-91de-301abc9a9cea None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:42.948873 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbe04e-8d42-4172-a26b-9ac38e8daca4 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:42.987077 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005276e-5cc3-4411-8d0a-d1205b77eeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:42.987820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005276e-5cc3-4411-8d0a-d1205b77eeaf None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:42.988104 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005276e-5cc3-4411-8d0a-d1205b77eeaf None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:42.988375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005276e-5cc3-4411-8d0a-d1205b77eeaf None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:42.989682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005276e-5cc3-4411-8d0a-d1205b77eeaf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:42.990050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005276e-5cc3-4411-8d0a-d1205b77eeaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:42.991470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005276e-5cc3-4411-8d0a-d1205b77eeaf None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:43.028528 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 259/507] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Aug 24 07:32:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:43.029456 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005276e-5cc3-4411-8d0a-d1205b77eeaf None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:43.045925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 249/508] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:43.045925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69537483-2a9d-41c4-91de-301abc9a9cea None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:43.056469 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:43.097421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:43.139229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005276e-5cc3-4411-8d0a-d1205b77eeaf None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:43.148400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69537483-2a9d-41c4-91de-301abc9a9cea None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:43.173028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436bdc6-7814-4d0d-ac4f-68d592a28be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:43.173655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436bdc6-7814-4d0d-ac4f-68d592a28be2 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:43.173941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436bdc6-7814-4d0d-ac4f-68d592a28be2 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:43.174240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436bdc6-7814-4d0d-ac4f-68d592a28be2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:43.181469 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8952ee-dc7d-4641-95c6-46d25ba02714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:43.181597 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436bdc6-7814-4d0d-ac4f-68d592a28be2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:43.182074 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8952ee-dc7d-4641-95c6-46d25ba02714 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:43.182356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8952ee-dc7d-4641-95c6-46d25ba02714 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:43.182627 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8952ee-dc7d-4641-95c6-46d25ba02714 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:43.182973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436bdc6-7814-4d0d-ac4f-68d592a28be2 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:43.183282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8952ee-dc7d-4641-95c6-46d25ba02714 None admin] RBAC: Adding query filter params (name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:43.183585 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8952ee-dc7d-4641-95c6-46d25ba02714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:43.184982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8952ee-dc7d-4641-95c6-46d25ba02714 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:43.192857 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 250/509] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:43.193642 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8952ee-dc7d-4641-95c6-46d25ba02714 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-4436bdc6-7814-4d0d-ac4f-68d592a28be2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:32:43.194111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:43.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:43.195849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:43.195849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:43.195849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:32:43.195849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:32:43.195849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:32:43.195849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:32:43.195849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:32:43.195849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:32:43.195849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:32:43.195849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:43.195849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 260/510] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:43.196413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436bdc6-7814-4d0d-ac4f-68d592a28be2 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:43.263594 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:43.294350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436bdc6-7814-4d0d-ac4f-68d592a28be2 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:43.294505 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8952ee-dc7d-4641-95c6-46d25ba02714 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:43.310842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:43.327254 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12200106-b9df-4bcf-8f9b-db4a6243be7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:43.327873 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12200106-b9df-4bcf-8f9b-db4a6243be7a None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:43.328153 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12200106-b9df-4bcf-8f9b-db4a6243be7a None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:43.328425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12200106-b9df-4bcf-8f9b-db4a6243be7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:43.328675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa593f-4c03-4807-9e49-87a7c0b43ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:43.329277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa593f-4c03-4807-9e49-87a7c0b43ce0 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:43.329562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa593f-4c03-4807-9e49-87a7c0b43ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:43.329838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa593f-4c03-4807-9e49-87a7c0b43ce0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:43.339032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12200106-b9df-4bcf-8f9b-db4a6243be7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:43.340586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12200106-b9df-4bcf-8f9b-db4a6243be7a None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-12200106-b9df-4bcf-8f9b-db4a6243be7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:43.355748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:43.356596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:43.357473 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:43.357473 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:43.357473 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:43.357473 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 261/511] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:43.357683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12200106-b9df-4bcf-8f9b-db4a6243be7a None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:43.401882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa593f-4c03-4807-9e49-87a7c0b43ce0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:43.403340 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa593f-4c03-4807-9e49-87a7c0b43ce0 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:43.454741 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12200106-b9df-4bcf-8f9b-db4a6243be7a None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:43.469153 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:43.488051 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890bdb0-7a17-4fff-97d9-fa82ab6d0ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:43.488775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890bdb0-7a17-4fff-97d9-fa82ab6d0ffc None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:43.489018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890bdb0-7a17-4fff-97d9-fa82ab6d0ffc None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:43.489293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890bdb0-7a17-4fff-97d9-fa82ab6d0ffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:43.499545 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890bdb0-7a17-4fff-97d9-fa82ab6d0ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:43.499781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 251/512] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:43.500600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa593f-4c03-4807-9e49-87a7c0b43ce0 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:43.501042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890bdb0-7a17-4fff-97d9-fa82ab6d0ffc None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-6890bdb0-7a17-4fff-97d9-fa82ab6d0ffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:43.515717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:43.516842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:43.517823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:43.517823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:43.517823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:43.517823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 262/513] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:43.517823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890bdb0-7a17-4fff-97d9-fa82ab6d0ffc None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:43.541864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:43.607615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa593f-4c03-4807-9e49-87a7c0b43ce0 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:43.616539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890bdb0-7a17-4fff-97d9-fa82ab6d0ffc None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:43.641063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e30b8-69c6-4044-a973-ac232c58f399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:43.641678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e30b8-69c6-4044-a973-ac232c58f399 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:43.641954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e30b8-69c6-4044-a973-ac232c58f399 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:43.642222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e30b8-69c6-4044-a973-ac232c58f399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:43.650413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b511ab9-4e37-4159-bb9f-c1ec19329613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:43.651015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b511ab9-4e37-4159-bb9f-c1ec19329613 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:43.651299 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b511ab9-4e37-4159-bb9f-c1ec19329613 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:43.651567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b511ab9-4e37-4159-bb9f-c1ec19329613 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:43.652284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b511ab9-4e37-4159-bb9f-c1ec19329613 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:43.652591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b511ab9-4e37-4159-bb9f-c1ec19329613 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:43.652882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e30b8-69c6-4044-a973-ac232c58f399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:43.654033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b511ab9-4e37-4159-bb9f-c1ec19329613 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:43.654314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e30b8-69c6-4044-a973-ac232c58f399 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-487e30b8-69c6-4044-a973-ac232c58f399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:43.669001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:43.669917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:43.670755 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:43.670755 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:43.670755 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:43.670755 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 252/514] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:43.670976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 263/515] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Aug 24 07:32:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:43.671800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b511ab9-4e37-4159-bb9f-c1ec19329613 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:43.671998 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e30b8-69c6-4044-a973-ac232c58f399 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:43.699214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:43.752216 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:43.771303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b511ab9-4e37-4159-bb9f-c1ec19329613 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:43.774241 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e30b8-69c6-4044-a973-ac232c58f399 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:43.806036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c255f3-95db-453a-8803-deb59c7aaf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:43.806659 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c255f3-95db-453a-8803-deb59c7aaf41 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:43.806941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c255f3-95db-453a-8803-deb59c7aaf41 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:43.807215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c255f3-95db-453a-8803-deb59c7aaf41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:43.807986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4c539-98bc-49d9-826a-1a936e369a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:43.808582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4c539-98bc-49d9-826a-1a936e369a14 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:43.808975 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4c539-98bc-49d9-826a-1a936e369a14 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:43.809157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4c539-98bc-49d9-826a-1a936e369a14 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:43.809788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4c539-98bc-49d9-826a-1a936e369a14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:43.810089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4c539-98bc-49d9-826a-1a936e369a14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:43.811454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4c539-98bc-49d9-826a-1a936e369a14 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:43.817039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c255f3-95db-453a-8803-deb59c7aaf41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:43.818455 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c255f3-95db-453a-8803-deb59c7aaf41 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:43.823203 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 253/516] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:43.823988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4c539-98bc-49d9-826a-1a936e369a14 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-89c255f3-95db-453a-8803-deb59c7aaf41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:43.836285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:43.837356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:43.838178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:43.838178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:43.838178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:43.838178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 264/517] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:43.838178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c255f3-95db-453a-8803-deb59c7aaf41 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:43.904209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:43.922490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4c539-98bc-49d9-826a-1a936e369a14 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:43.937391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c255f3-95db-453a-8803-deb59c7aaf41 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:43.955975 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883556a-8d28-4925-9270-e879f3eb8775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:43.956582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883556a-8d28-4925-9270-e879f3eb8775 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:43.956892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883556a-8d28-4925-9270-e879f3eb8775 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:43.957170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883556a-8d28-4925-9270-e879f3eb8775 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:43.962409 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2883556a-8d28-4925-9270-e879f3eb8775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:43.962723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2883556a-8d28-4925-9270-e879f3eb8775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:43.964111 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2883556a-8d28-4925-9270-e879f3eb8775 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:43.970086 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:43.976939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c62f1-58bc-407f-b5ec-704a49138b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:43.979474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 254/518] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:43.979573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c62f1-58bc-407f-b5ec-704a49138b84 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:43.979646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883556a-8d28-4925-9270-e879f3eb8775 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:43.979715 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c62f1-58bc-407f-b5ec-704a49138b84 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:43.979715 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c62f1-58bc-407f-b5ec-704a49138b84 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:43.979715 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c62f1-58bc-407f-b5ec-704a49138b84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:43.979715 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c62f1-58bc-407f-b5ec-704a49138b84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:43.980428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c62f1-58bc-407f-b5ec-704a49138b84 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:43.992355 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 265/519] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:43.993201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c62f1-58bc-407f-b5ec-704a49138b84 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:44.078899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883556a-8d28-4925-9270-e879f3eb8775 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:44.094353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c62f1-58bc-407f-b5ec-704a49138b84 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:44.114607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371ac03-6815-4d68-8418-420ca1cdfac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:44.115219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371ac03-6815-4d68-8418-420ca1cdfac2 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:44.115503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371ac03-6815-4d68-8418-420ca1cdfac2 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:44.115779 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371ac03-6815-4d68-8418-420ca1cdfac2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:44.127311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:44.127785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d13239-f3b1-4600-81ce-8b53215d76b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:44.128410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d13239-f3b1-4600-81ce-8b53215d76b4 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:44.128717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d13239-f3b1-4600-81ce-8b53215d76b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:44.128992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d13239-f3b1-4600-81ce-8b53215d76b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:44.135553 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d13239-f3b1-4600-81ce-8b53215d76b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:44.137135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d13239-f3b1-4600-81ce-8b53215d76b4 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-87d13239-f3b1-4600-81ce-8b53215d76b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:32:44.149170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:44.150073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:44.150904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:32:44.150904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:32:44.150904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:32:44.150904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:32:44.150904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:32:44.150904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:32:44.150904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:44.150904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:44.150904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 266/520] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:44.151888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d13239-f3b1-4600-81ce-8b53215d76b4 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:44.183661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371ac03-6815-4d68-8418-420ca1cdfac2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:44.185094 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371ac03-6815-4d68-8418-420ca1cdfac2 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:44.204322 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:44.251075 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d13239-f3b1-4600-81ce-8b53215d76b4 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:44.282261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 255/521] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:44.283079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371ac03-6815-4d68-8418-420ca1cdfac2 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:44.283311 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3e7fe-ce57-4dd5-b8ff-4777c20e71ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:44.283816 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3e7fe-ce57-4dd5-b8ff-4777c20e71ec None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:44.284101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3e7fe-ce57-4dd5-b8ff-4777c20e71ec None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:44.284371 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3e7fe-ce57-4dd5-b8ff-4777c20e71ec None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:44.285036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3e7fe-ce57-4dd5-b8ff-4777c20e71ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:44.285344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3e7fe-ce57-4dd5-b8ff-4777c20e71ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:44.287236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3e7fe-ce57-4dd5-b8ff-4777c20e71ec None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:44.298892 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 267/522] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:44.299668 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3e7fe-ce57-4dd5-b8ff-4777c20e71ec None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:44.326888 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:44.382986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371ac03-6815-4d68-8418-420ca1cdfac2 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:44.398522 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3e7fe-ce57-4dd5-b8ff-4777c20e71ec None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:44.411337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:44.416637 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af5ba8-16d1-41d1-b7b2-616a97c38a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:44.417263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af5ba8-16d1-41d1-b7b2-616a97c38a0e None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:44.417546 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af5ba8-16d1-41d1-b7b2-616a97c38a0e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:44.417824 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af5ba8-16d1-41d1-b7b2-616a97c38a0e None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:44.418490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af5ba8-16d1-41d1-b7b2-616a97c38a0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:44.419966 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af5ba8-16d1-41d1-b7b2-616a97c38a0e None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:44.431510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75a2f7-f194-4f1d-8a7c-57e795ed6704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:44.432137 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75a2f7-f194-4f1d-8a7c-57e795ed6704 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:44.432416 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75a2f7-f194-4f1d-8a7c-57e795ed6704 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:44.432710 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75a2f7-f194-4f1d-8a7c-57e795ed6704 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:44.439829 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75a2f7-f194-4f1d-8a7c-57e795ed6704 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:44.441490 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75a2f7-f194-4f1d-8a7c-57e795ed6704 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-cc75a2f7-f194-4f1d-8a7c-57e795ed6704 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:32:44.453251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:44.454239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:44.455301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:44.455301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:44.455301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:32:44.455301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:32:44.455301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:32:44.455301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:32:44.455301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:32:44.455301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:32:44.455301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:32:44.455301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:44.455301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 268/523] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:44.455301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75a2f7-f194-4f1d-8a7c-57e795ed6704 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:44.525730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 256/524] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:32:44 2022] POST /identity/v3/users => generated 247 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:32:44.526658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af5ba8-16d1-41d1-b7b2-616a97c38a0e None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:44.568742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75a2f7-f194-4f1d-8a7c-57e795ed6704 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:44.611493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfc085-d062-4387-a06e-f88641a28f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:44.612183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfc085-d062-4387-a06e-f88641a28f52 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:44.612471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfc085-d062-4387-a06e-f88641a28f52 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:44.612780 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfc085-d062-4387-a06e-f88641a28f52 None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:44.614025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfc085-d062-4387-a06e-f88641a28f52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:44.614400 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfc085-d062-4387-a06e-f88641a28f52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:44.615954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfc085-d062-4387-a06e-f88641a28f52 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:44.635254 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 269/525] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:32:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:44.637659 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfc085-d062-4387-a06e-f88641a28f52 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:44.640141 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:44.645270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af5ba8-16d1-41d1-b7b2-616a97c38a0e None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:44.672871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:44.685184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b58298-2c94-494f-8852-9ac2fe629a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:44.685820 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b58298-2c94-494f-8852-9ac2fe629a1c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:44.686099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b58298-2c94-494f-8852-9ac2fe629a1c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:44.686370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b58298-2c94-494f-8852-9ac2fe629a1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:44.697302 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b58298-2c94-494f-8852-9ac2fe629a1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:44.698757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b58298-2c94-494f-8852-9ac2fe629a1c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-40b58298-2c94-494f-8852-9ac2fe629a1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:44.715009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:44.716020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:44.716945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:44.716945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:44.716945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:44.716945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 257/526] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:44.717966 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b58298-2c94-494f-8852-9ac2fe629a1c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:44.758660 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfc085-d062-4387-a06e-f88641a28f52 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:44.803773 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4023f-548a-453c-a2d6-1f4b842f7331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:44.804410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4023f-548a-453c-a2d6-1f4b842f7331 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:44.804716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4023f-548a-453c-a2d6-1f4b842f7331 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:44.804993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4023f-548a-453c-a2d6-1f4b842f7331 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:44.811813 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4023f-548a-453c-a2d6-1f4b842f7331 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:44.813299 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4023f-548a-453c-a2d6-1f4b842f7331 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:44.825459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b58298-2c94-494f-8852-9ac2fe629a1c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-21b4023f-548a-453c-a2d6-1f4b842f7331 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:32:44.825634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:44.826401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:44.827245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:32:44.827245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:32:44.827245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:32:44.827245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:32:44.827245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:32:44.827245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:32:44.827245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:44.827245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:44.827245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 270/527] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:44.827704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4023f-548a-453c-a2d6-1f4b842f7331 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:44.867326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd506e1-6bc0-4d7d-9f7f-27e79afa5fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:44.867929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd506e1-6bc0-4d7d-9f7f-27e79afa5fd3 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:44.868213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd506e1-6bc0-4d7d-9f7f-27e79afa5fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:44.868565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd506e1-6bc0-4d7d-9f7f-27e79afa5fd3 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:44.869385 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd506e1-6bc0-4d7d-9f7f-27e79afa5fd3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:44.869694 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd506e1-6bc0-4d7d-9f7f-27e79afa5fd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:44.871207 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd506e1-6bc0-4d7d-9f7f-27e79afa5fd3 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:44.889832 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 258/528] 10.222.0.26 () {56 vars in 1139 bytes} [Wed Aug 24 07:32:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:44.893850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd506e1-6bc0-4d7d-9f7f-27e79afa5fd3 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:44.905651 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:44.925706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:44.935718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4023f-548a-453c-a2d6-1f4b842f7331 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:44.973925 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5625438-017d-445f-98f0-cb2b0f6b3cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:44.974611 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5625438-017d-445f-98f0-cb2b0f6b3cfc None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:44.974899 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5625438-017d-445f-98f0-cb2b0f6b3cfc None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:44.975180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5625438-017d-445f-98f0-cb2b0f6b3cfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:44.976400 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5625438-017d-445f-98f0-cb2b0f6b3cfc None admin] RBAC: Adding query filter params (role.id=3acdf08298c84325b53a0e286cc53b30, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=bdf81f09006a42299c0807f77f16c748) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:44.976742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5625438-017d-445f-98f0-cb2b0f6b3cfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:44.978565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5625438-017d-445f-98f0-cb2b0f6b3cfc None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:44.985045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 271/529] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:32:44 2022] GET /identity/v3/role_assignments?user.id=bdf81f09006a42299c0807f77f16c748&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=3acdf08298c84325b53a0e286cc53b30 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:44.985848 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5625438-017d-445f-98f0-cb2b0f6b3cfc None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:45.013344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd506e1-6bc0-4d7d-9f7f-27e79afa5fd3 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:45.062742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd078ad8-2f6f-4735-9b00-27189e2ed71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:45.063373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd078ad8-2f6f-4735-9b00-27189e2ed71a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:45.063659 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd078ad8-2f6f-4735-9b00-27189e2ed71a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:45.063938 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd078ad8-2f6f-4735-9b00-27189e2ed71a None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:45.064595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd078ad8-2f6f-4735-9b00-27189e2ed71a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:45.064935 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd078ad8-2f6f-4735-9b00-27189e2ed71a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:45.066483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd078ad8-2f6f-4735-9b00-27189e2ed71a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:45.080463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 259/530] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:45.081478 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd078ad8-2f6f-4735-9b00-27189e2ed71a None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:45.125747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5625438-017d-445f-98f0-cb2b0f6b3cfc None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:45.165915 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:45.168088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72765c75-28da-4abc-ae25-2dfb83b4eda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:45.168766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72765c75-28da-4abc-ae25-2dfb83b4eda6 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:45.169052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72765c75-28da-4abc-ae25-2dfb83b4eda6 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:45.169363 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72765c75-28da-4abc-ae25-2dfb83b4eda6 None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:45.170017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72765c75-28da-4abc-ae25-2dfb83b4eda6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:45.170326 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72765c75-28da-4abc-ae25-2dfb83b4eda6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:45.171751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72765c75-28da-4abc-ae25-2dfb83b4eda6 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:45.185100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 272/531] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:32:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:45.185963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72765c75-28da-4abc-ae25-2dfb83b4eda6 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:45.205830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd078ad8-2f6f-4735-9b00-27189e2ed71a None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:45.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:45.249341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38474b12-2885-48fa-b6cf-063e1613176e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:45.249958 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38474b12-2885-48fa-b6cf-063e1613176e None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:45.250244 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38474b12-2885-48fa-b6cf-063e1613176e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:45.250523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38474b12-2885-48fa-b6cf-063e1613176e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:45.302962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72765c75-28da-4abc-ae25-2dfb83b4eda6 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:45.337666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38474b12-2885-48fa-b6cf-063e1613176e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:45.339198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38474b12-2885-48fa-b6cf-063e1613176e None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:45.342280 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767988d-29b7-4903-8aa7-cd0e38469d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:45.343205 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767988d-29b7-4903-8aa7-cd0e38469d7c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:45.343694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767988d-29b7-4903-8aa7-cd0e38469d7c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:45.344160 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767988d-29b7-4903-8aa7-cd0e38469d7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:45.357426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767988d-29b7-4903-8aa7-cd0e38469d7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:45.358886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767988d-29b7-4903-8aa7-cd0e38469d7c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-b767988d-29b7-4903-8aa7-cd0e38469d7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:32:45.373337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:45.374218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:45.375053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:32:45.375053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:32:45.375053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:32:45.375053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:32:45.375053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:32:45.375053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:32:45.375053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:45.375053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:45.375053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 273/532] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:45.375957 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767988d-29b7-4903-8aa7-cd0e38469d7c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:45.436369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:45.465394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:45.475811 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 260/533] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:45.476493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38474b12-2885-48fa-b6cf-063e1613176e None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:45.501405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767988d-29b7-4903-8aa7-cd0e38469d7c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:45.544064 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405df004-407c-431d-a6a7-9268847a72ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:45.544935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405df004-407c-431d-a6a7-9268847a72ba None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:45.545291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405df004-407c-431d-a6a7-9268847a72ba None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:45.545637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405df004-407c-431d-a6a7-9268847a72ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:45.546459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405df004-407c-431d-a6a7-9268847a72ba None admin] RBAC: Adding query filter params (role.id=3acdf08298c84325b53a0e286cc53b30, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=43e57131a5be4af785166bc03a3337a2) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:45.546927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405df004-407c-431d-a6a7-9268847a72ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:45.548365 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405df004-407c-431d-a6a7-9268847a72ba None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:45.555646 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 274/534] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:32:45 2022] GET /identity/v3/role_assignments?user.id=43e57131a5be4af785166bc03a3337a2&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=3acdf08298c84325b53a0e286cc53b30 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:45.556634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405df004-407c-431d-a6a7-9268847a72ba None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:45.616553 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38474b12-2885-48fa-b6cf-063e1613176e None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:45.666476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f212d-e1e7-400b-ace7-3de093f3c746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:45.668026 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f212d-e1e7-400b-ace7-3de093f3c746 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:45.668491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f212d-e1e7-400b-ace7-3de093f3c746 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:45.668982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f212d-e1e7-400b-ace7-3de093f3c746 None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:45.669989 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f212d-e1e7-400b-ace7-3de093f3c746 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:45.670529 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f212d-e1e7-400b-ace7-3de093f3c746 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:45.672185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f212d-e1e7-400b-ace7-3de093f3c746 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:45.684697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405df004-407c-431d-a6a7-9268847a72ba None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:45.708124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 261/535] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:32:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:45.713825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f212d-e1e7-400b-ace7-3de093f3c746 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:45.728202 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd2ca0-da6f-4fdd-b7ad-bf813c11fbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:45.728889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd2ca0-da6f-4fdd-b7ad-bf813c11fbb4 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:45.729220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd2ca0-da6f-4fdd-b7ad-bf813c11fbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:45.729488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd2ca0-da6f-4fdd-b7ad-bf813c11fbb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:45.745057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd2ca0-da6f-4fdd-b7ad-bf813c11fbb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:45.745247 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd2ca0-da6f-4fdd-b7ad-bf813c11fbb4 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:45.745880 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-d3bd2ca0-da6f-4fdd-b7ad-bf813c11fbb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:45.767459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:45.768636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:45.769505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:45.769505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:45.769505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:45.769505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 275/536] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:45.769745 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd2ca0-da6f-4fdd-b7ad-bf813c11fbb4 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:45.798084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:45.916407 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd2ca0-da6f-4fdd-b7ad-bf813c11fbb4 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:45.950325 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f212d-e1e7-400b-ace7-3de093f3c746 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:45.987051 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421539d-76ac-4c38-9c7e-d62666305422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:45.987685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421539d-76ac-4c38-9c7e-d62666305422 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:45.988004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421539d-76ac-4c38-9c7e-d62666305422 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:45.988270 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421539d-76ac-4c38-9c7e-d62666305422 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:45.993298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421539d-76ac-4c38-9c7e-d62666305422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:45.993598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421539d-76ac-4c38-9c7e-d62666305422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:45.994990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421539d-76ac-4c38-9c7e-d62666305422 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:46.026772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 276/537] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:46.027338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744b7fc-16d5-4298-a7d7-79f29b9c8404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:46.027950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744b7fc-16d5-4298-a7d7-79f29b9c8404 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:46.028229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744b7fc-16d5-4298-a7d7-79f29b9c8404 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:46.028502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744b7fc-16d5-4298-a7d7-79f29b9c8404 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:46.029344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744b7fc-16d5-4298-a7d7-79f29b9c8404 None admin] RBAC: Adding query filter params (role.id=3acdf08298c84325b53a0e286cc53b30, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=38de805ba5eb4aa58cf95d2082c2d216) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:46.029650 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744b7fc-16d5-4298-a7d7-79f29b9c8404 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:46.031133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744b7fc-16d5-4298-a7d7-79f29b9c8404 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:46.038180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421539d-76ac-4c38-9c7e-d62666305422 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:46.042508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 262/538] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:32:45 2022] GET /identity/v3/role_assignments?user.id=38de805ba5eb4aa58cf95d2082c2d216&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=3acdf08298c84325b53a0e286cc53b30 => generated 259 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:46.097836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:46.162141 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421539d-76ac-4c38-9c7e-d62666305422 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:46.211679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c66d26-b39a-453f-889d-b74d5046ab15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:46.212343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c66d26-b39a-453f-889d-b74d5046ab15 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:46.212624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c66d26-b39a-453f-889d-b74d5046ab15 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:46.212922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c66d26-b39a-453f-889d-b74d5046ab15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:46.221078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c66d26-b39a-453f-889d-b74d5046ab15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:46.222547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c66d26-b39a-453f-889d-b74d5046ab15 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-53c66d26-b39a-453f-889d-b74d5046ab15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:32:46.237178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:46.238063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:46.239136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:32:46.239136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:32:46.239136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:32:46.239136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:32:46.239136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:32:46.239136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:32:46.239136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:46.239136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:46.239136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 277/539] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:32:46.246209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744b7fc-16d5-4298-a7d7-79f29b9c8404 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:46.368417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:46.377438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744b7fc-16d5-4298-a7d7-79f29b9c8404 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:46.428120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53be5f2-5229-48b7-a9c2-5b3b2ccdb33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:46.428911 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53be5f2-5229-48b7-a9c2-5b3b2ccdb33e None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:46.429205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53be5f2-5229-48b7-a9c2-5b3b2ccdb33e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:46.429479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53be5f2-5229-48b7-a9c2-5b3b2ccdb33e None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:46.430169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53be5f2-5229-48b7-a9c2-5b3b2ccdb33e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:46.430476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53be5f2-5229-48b7-a9c2-5b3b2ccdb33e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:46.431916 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53be5f2-5229-48b7-a9c2-5b3b2ccdb33e None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:46.451969 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 263/540] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:32:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:46.458523 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c66d26-b39a-453f-889d-b74d5046ab15 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:46.574253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c66d26-b39a-453f-889d-b74d5046ab15 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:46.587290 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:46.619498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0e9d9-0bd2-45ca-8441-d0df97e784e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:46.620032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0e9d9-0bd2-45ca-8441-d0df97e784e5 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:46.620315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0e9d9-0bd2-45ca-8441-d0df97e784e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:46.620587 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0e9d9-0bd2-45ca-8441-d0df97e784e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:46.621336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0e9d9-0bd2-45ca-8441-d0df97e784e5 None admin] RBAC: Adding query filter params (role.id=3acdf08298c84325b53a0e286cc53b30, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=46e61402c01c4f9ba5f18e94960bc313) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:46.621646 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0e9d9-0bd2-45ca-8441-d0df97e784e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:46.623102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0e9d9-0bd2-45ca-8441-d0df97e784e5 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:46.633044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 278/541] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:32:46 2022] GET /identity/v3/role_assignments?user.id=46e61402c01c4f9ba5f18e94960bc313&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=3acdf08298c84325b53a0e286cc53b30 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:46.645392 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b39a97-2b26-4b29-a5f8-0e0fcd765285 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:46.645737 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b39a97-2b26-4b29-a5f8-0e0fcd765285 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:46.645950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b39a97-2b26-4b29-a5f8-0e0fcd765285 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:46.647333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 264/542] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:46.653232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb19a2-1ae9-4c88-867a-b85da0477f11 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:46.653520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb19a2-1ae9-4c88-867a-b85da0477f11 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:46.653804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb19a2-1ae9-4c88-867a-b85da0477f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:46.680022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-70cb19a2-1ae9-4c88-867a-b85da0477f11 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:46.681283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-70cb19a2-1ae9-4c88-867a-b85da0477f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:46.770188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 279/543] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:32:47.173439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b39a97-2b26-4b29-a5f8-0e0fcd765285 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:47.319012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b39a97-2b26-4b29-a5f8-0e0fcd765285 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:47.369561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23820b32-4624-4b3b-9547-aa6f43787224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:47.370556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23820b32-4624-4b3b-9547-aa6f43787224 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:47.371040 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23820b32-4624-4b3b-9547-aa6f43787224 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:47.371521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23820b32-4624-4b3b-9547-aa6f43787224 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:47.393681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23820b32-4624-4b3b-9547-aa6f43787224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-23820b32-4624-4b3b-9547-aa6f43787224 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:47.399424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:47.400309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:47.401126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:47.401126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:47.401126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:47.401126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:47.401126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:47.401126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:47.401126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:47.401126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:47.401126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:47.401126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:47.401126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:47.401126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:47.403353 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 265/544] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:47.409175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb19a2-1ae9-4c88-867a-b85da0477f11 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:47.547245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb19a2-1ae9-4c88-867a-b85da0477f11 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:47.556692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:47.599239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfac33-6aa9-434e-8901-e65d776a192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:47.599858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfac33-6aa9-434e-8901-e65d776a192f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:47.600140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfac33-6aa9-434e-8901-e65d776a192f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:47.600412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfac33-6aa9-434e-8901-e65d776a192f None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:47.601118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfac33-6aa9-434e-8901-e65d776a192f None admin] RBAC: Adding query filter params (name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:47.601423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfac33-6aa9-434e-8901-e65d776a192f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:47.602851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfac33-6aa9-434e-8901-e65d776a192f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:47.617000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 280/545] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:47.617390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23820b32-4624-4b3b-9547-aa6f43787224 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:47.622207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a74e8d-6ca8-4030-af0b-51ac57313c9b None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:47.622431 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a74e8d-6ca8-4030-af0b-51ac57313c9b None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:47.623158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a74e8d-6ca8-4030-af0b-51ac57313c9b None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:47.625244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 281/546] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:32:47.636380 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931301-b1f9-46b7-b716-d2ab659982ab None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:47.636683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931301-b1f9-46b7-b716-d2ab659982ab None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:47.636964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931301-b1f9-46b7-b716-d2ab659982ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:47.649567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:47.663119 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-6b931301-b1f9-46b7-b716-d2ab659982ab None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:47.664322 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6b931301-b1f9-46b7-b716-d2ab659982ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:47.749297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23820b32-4624-4b3b-9547-aa6f43787224 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:47.788489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f00c8-3cb3-4885-a488-42a7c668c020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:47.789066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f00c8-3cb3-4885-a488-42a7c668c020 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:47.789388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f00c8-3cb3-4885-a488-42a7c668c020 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:47.789670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f00c8-3cb3-4885-a488-42a7c668c020 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:47.800551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f00c8-3cb3-4885-a488-42a7c668c020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:47.802663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f00c8-3cb3-4885-a488-42a7c668c020 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-740f00c8-3cb3-4885-a488-42a7c668c020 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:47.817681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:47.818697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:47.819556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:47.819556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:47.819556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:47.823610 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 266/547] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:32:47.824909 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f00c8-3cb3-4885-a488-42a7c668c020 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:47.826278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 282/548] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:32:47.872598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce57b5-8591-4c21-9ab9-494346f29366 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:47.872958 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce57b5-8591-4c21-9ab9-494346f29366 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:47.873475 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce57b5-8591-4c21-9ab9-494346f29366 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:47.874659 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 283/549] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:32:47.880071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7b8cd-7ba6-4c02-ae2d-91d727428d62 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:47.880363 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7b8cd-7ba6-4c02-ae2d-91d727428d62 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:47.880663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7b8cd-7ba6-4c02-ae2d-91d727428d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:47.898729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:47.910020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-14a7b8cd-7ba6-4c02-ae2d-91d727428d62 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:47.911259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-14a7b8cd-7ba6-4c02-ae2d-91d727428d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:47.930884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f00c8-3cb3-4885-a488-42a7c668c020 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:47.966926 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb7dc3-4a35-4471-8036-a6114c36ca5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:47.971005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb7dc3-4a35-4471-8036-a6114c36ca5f None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:47.971292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb7dc3-4a35-4471-8036-a6114c36ca5f None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:47.971583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb7dc3-4a35-4471-8036-a6114c36ca5f None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:47.972275 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb7dc3-4a35-4471-8036-a6114c36ca5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:47.973479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb7dc3-4a35-4471-8036-a6114c36ca5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:47.975554 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb7dc3-4a35-4471-8036-a6114c36ca5f None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:48.005417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 267/550] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:48.010103 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb7dc3-4a35-4471-8036-a6114c36ca5f None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:48.024854 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 284/551] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:48.148692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb7dc3-4a35-4471-8036-a6114c36ca5f None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:48.152841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7b8cd-7ba6-4c02-ae2d-91d727428d62 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:48.174621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:48.185785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9444b9-795a-4887-8b4f-5dbab1faae58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:48.186480 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9444b9-795a-4887-8b4f-5dbab1faae58 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:48.193029 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9444b9-795a-4887-8b4f-5dbab1faae58 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:48.193029 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9444b9-795a-4887-8b4f-5dbab1faae58 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:48.196180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9444b9-795a-4887-8b4f-5dbab1faae58 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:48.197721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9444b9-795a-4887-8b4f-5dbab1faae58 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-ed9444b9-795a-4887-8b4f-5dbab1faae58 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:32:48.213056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:48.214054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:48.214971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:48.214971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:48.214971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:32:48.214971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:32:48.214971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:32:48.214971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:32:48.214971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:32:48.214971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:32:48.214971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:32:48.214971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:48.218894 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 268/552] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:32:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:32:48.219659 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:48.220278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9444b9-795a-4887-8b4f-5dbab1faae58 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:48.327618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9444b9-795a-4887-8b4f-5dbab1faae58 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:48.355300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7b8cd-7ba6-4c02-ae2d-91d727428d62 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:48.371419 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1cc32-0257-48d2-8eae-ac7cb4b2f535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:48.372629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1cc32-0257-48d2-8eae-ac7cb4b2f535 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:48.372931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1cc32-0257-48d2-8eae-ac7cb4b2f535 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:48.373200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1cc32-0257-48d2-8eae-ac7cb4b2f535 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:48.373922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1cc32-0257-48d2-8eae-ac7cb4b2f535 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:48.374343 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1cc32-0257-48d2-8eae-ac7cb4b2f535 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:48.375782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1cc32-0257-48d2-8eae-ac7cb4b2f535 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:48.392024 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481bcc8-b31e-4d63-bb9a-ae1b7820e8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:48.392680 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481bcc8-b31e-4d63-bb9a-ae1b7820e8ea None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:48.392970 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481bcc8-b31e-4d63-bb9a-ae1b7820e8ea None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:48.393244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481bcc8-b31e-4d63-bb9a-ae1b7820e8ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:48.395382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 269/553] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Aug 24 07:32:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:48.396769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b42c6f-94cd-41d7-a8c6-c0bad4709ebe None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:48.397057 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b42c6f-94cd-41d7-a8c6-c0bad4709ebe None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:48.397327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b42c6f-94cd-41d7-a8c6-c0bad4709ebe None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:48.402532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 270/554] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:48.402532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b42c6f-94cd-41d7-a8c6-c0bad4709ebe None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:48.410204 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481bcc8-b31e-4d63-bb9a-ae1b7820e8ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-d481bcc8-b31e-4d63-bb9a-ae1b7820e8ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:48.417131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:48.418031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:48.418919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:48.418919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:48.418919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:48.418919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:48.418919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:48.418919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:48.418919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:48.418919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:48.418919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:48.418919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:48.418919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:48.418919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:48.419552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 285/555] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:48.424867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481bcc8-b31e-4d63-bb9a-ae1b7820e8ea None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:48.446749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:48.509225 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b42c6f-94cd-41d7-a8c6-c0bad4709ebe None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:48.517843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:48.533731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481bcc8-b31e-4d63-bb9a-ae1b7820e8ea None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:48.544581 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc1bdd-5946-4604-9065-5d0caf92b18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:48.545237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc1bdd-5946-4604-9065-5d0caf92b18d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:48.545518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc1bdd-5946-4604-9065-5d0caf92b18d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:48.545789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc1bdd-5946-4604-9065-5d0caf92b18d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:48.557411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc1bdd-5946-4604-9065-5d0caf92b18d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-efdc1bdd-5946-4604-9065-5d0caf92b18d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:48.562365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:48.563462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:48.564476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:48.564476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:48.564476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:48.564476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:48.564476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:48.564476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:48.564476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:48.564476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:48.564476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:48.564476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:48.564476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:48.564476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:48.571833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 271/556] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:48.571833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750ef80-729b-43de-a8ec-0a3f13aaca83 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:48.571833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750ef80-729b-43de-a8ec-0a3f13aaca83 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:48.571833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750ef80-729b-43de-a8ec-0a3f13aaca83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:48.574767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b57552-5d79-4fe8-b41e-4b75594e40de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:48.575372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b57552-5d79-4fe8-b41e-4b75594e40de None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:48.575649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b57552-5d79-4fe8-b41e-4b75594e40de None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:48.575915 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b57552-5d79-4fe8-b41e-4b75594e40de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:48.588570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-0750ef80-729b-43de-a8ec-0a3f13aaca83 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:48.589817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0750ef80-729b-43de-a8ec-0a3f13aaca83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:48.649675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b57552-5d79-4fe8-b41e-4b75594e40de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:48.651157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b57552-5d79-4fe8-b41e-4b75594e40de None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:48.665623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 272/557] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:48.666468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750ef80-729b-43de-a8ec-0a3f13aaca83 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:48.706407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:48.758445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:48.776568 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 286/558] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:48.776568 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b57552-5d79-4fe8-b41e-4b75594e40de None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:48.783848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750ef80-729b-43de-a8ec-0a3f13aaca83 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:48.820083 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83801e-79e6-49a9-9fc8-26ba2dad38ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:48.820970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83801e-79e6-49a9-9fc8-26ba2dad38ee None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:48.821400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83801e-79e6-49a9-9fc8-26ba2dad38ee None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:48.821858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83801e-79e6-49a9-9fc8-26ba2dad38ee None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:48.822670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83801e-79e6-49a9-9fc8-26ba2dad38ee None admin] RBAC: Adding query filter params (name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:48.823203 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83801e-79e6-49a9-9fc8-26ba2dad38ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:48.824734 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83801e-79e6-49a9-9fc8-26ba2dad38ee None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:48.833503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 273/559] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:48.834548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83801e-79e6-49a9-9fc8-26ba2dad38ee None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:48.880074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b57552-5d79-4fe8-b41e-4b75594e40de None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:48.916391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8433f-3dd9-441e-9e56-734c7b880546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:48.917013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8433f-3dd9-441e-9e56-734c7b880546 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:48.917311 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8433f-3dd9-441e-9e56-734c7b880546 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:48.917508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8433f-3dd9-441e-9e56-734c7b880546 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:48.918309 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8433f-3dd9-441e-9e56-734c7b880546 None admin] RBAC: Adding query filter params (name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:48.918608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8433f-3dd9-441e-9e56-734c7b880546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:48.920025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8433f-3dd9-441e-9e56-734c7b880546 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:48.932803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 287/560] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:48.932803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8433f-3dd9-441e-9e56-734c7b880546 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:48.939046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:48.948672 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83801e-79e6-49a9-9fc8-26ba2dad38ee None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:48.988100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc3b84-0d42-41d6-9f5f-70845c4b90c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:48.988864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3b84-0d42-41d6-9f5f-70845c4b90c8 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:48.989148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3b84-0d42-41d6-9f5f-70845c4b90c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:48.989484 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3b84-0d42-41d6-9f5f-70845c4b90c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:48.996056 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:48.999794 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc3b84-0d42-41d6-9f5f-70845c4b90c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:49.001358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc3b84-0d42-41d6-9f5f-70845c4b90c8 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-76bc3b84-0d42-41d6-9f5f-70845c4b90c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:49.016972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:49.017945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:49.018834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:49.018834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:49.018834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:49.018834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 274/561] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:49.018834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc3b84-0d42-41d6-9f5f-70845c4b90c8 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:49.055767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8433f-3dd9-441e-9e56-734c7b880546 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:49.089167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0765c-26ab-4e72-8883-68df755115d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:49.089793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0765c-26ab-4e72-8883-68df755115d6 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.090075 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0765c-26ab-4e72-8883-68df755115d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.090349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0765c-26ab-4e72-8883-68df755115d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.100862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af0765c-26ab-4e72-8883-68df755115d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:49.102307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af0765c-26ab-4e72-8883-68df755115d6 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-8af0765c-26ab-4e72-8883-68df755115d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:49.116748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:49.117886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:49.118983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:49.118983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:49.118983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:49.118983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 288/562] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:49.118983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0765c-26ab-4e72-8883-68df755115d6 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:49.128665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc3b84-0d42-41d6-9f5f-70845c4b90c8 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:49.163592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30657a41-1758-4e12-9c59-51ec867a9db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:49.164215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30657a41-1758-4e12-9c59-51ec867a9db0 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.164504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30657a41-1758-4e12-9c59-51ec867a9db0 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.164800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30657a41-1758-4e12-9c59-51ec867a9db0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.175222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30657a41-1758-4e12-9c59-51ec867a9db0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:49.176823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30657a41-1758-4e12-9c59-51ec867a9db0 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:49.183121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-30657a41-1758-4e12-9c59-51ec867a9db0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:49.190986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:49.195081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:49.195994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:49.195994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:49.195994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:49.195994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 275/563] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:49.195994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30657a41-1758-4e12-9c59-51ec867a9db0 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:49.218586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0765c-26ab-4e72-8883-68df755115d6 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:49.220094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:49.253692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133005f-e07b-467a-8d55-0c6f49ebcfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:49.254551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133005f-e07b-467a-8d55-0c6f49ebcfd9 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.255001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133005f-e07b-467a-8d55-0c6f49ebcfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.255279 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133005f-e07b-467a-8d55-0c6f49ebcfd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.265295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133005f-e07b-467a-8d55-0c6f49ebcfd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-0133005f-e07b-467a-8d55-0c6f49ebcfd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:49.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:49.271552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:49.272396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:49.272396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:49.272396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:49.272396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:49.272396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:49.272396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:49.272396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:49.272396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:49.272396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:49.272396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:49.272396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:49.272396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:49.272396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 289/564] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:49.273529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404de94-884e-40cc-b2ce-9fc1444027ef None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.273529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404de94-884e-40cc-b2ce-9fc1444027ef None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.273664 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404de94-884e-40cc-b2ce-9fc1444027ef None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.275010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 290/565] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:49.275625 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404de94-884e-40cc-b2ce-9fc1444027ef None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:49.300249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30657a41-1758-4e12-9c59-51ec867a9db0 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:49.333194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbb119-082b-4987-b591-82ab0b1b93dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:49.333787 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbb119-082b-4987-b591-82ab0b1b93dc None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.334071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbb119-082b-4987-b591-82ab0b1b93dc None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.334351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbb119-082b-4987-b591-82ab0b1b93dc None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.334981 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbb119-082b-4987-b591-82ab0b1b93dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:49.335289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbb119-082b-4987-b591-82ab0b1b93dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:49.336676 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbb119-082b-4987-b591-82ab0b1b93dc None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:49.349280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 276/566] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:49.350276 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbb119-082b-4987-b591-82ab0b1b93dc None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:49.380571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404de94-884e-40cc-b2ce-9fc1444027ef None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:49.406399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:49.415224 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266b0c7-46e2-4c70-a5e9-fd4e7a959367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:49.415822 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266b0c7-46e2-4c70-a5e9-fd4e7a959367 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.416106 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266b0c7-46e2-4c70-a5e9-fd4e7a959367 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.416383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266b0c7-46e2-4c70-a5e9-fd4e7a959367 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.417061 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266b0c7-46e2-4c70-a5e9-fd4e7a959367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:49.422634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266b0c7-46e2-4c70-a5e9-fd4e7a959367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:49.422634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266b0c7-46e2-4c70-a5e9-fd4e7a959367 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:49.430812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 291/567] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:49.431445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266b0c7-46e2-4c70-a5e9-fd4e7a959367 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:49.447039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbb119-082b-4987-b591-82ab0b1b93dc None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:49.459955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:49.482073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53837bdb-70da-4421-88dd-3a9c6f7f0bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:49.482680 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53837bdb-70da-4421-88dd-3a9c6f7f0bc2 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.482967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53837bdb-70da-4421-88dd-3a9c6f7f0bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.483241 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53837bdb-70da-4421-88dd-3a9c6f7f0bc2 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.483881 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53837bdb-70da-4421-88dd-3a9c6f7f0bc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:49.484189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53837bdb-70da-4421-88dd-3a9c6f7f0bc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:49.485586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53837bdb-70da-4421-88dd-3a9c6f7f0bc2 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:49.496851 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 277/568] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:49.497995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c2ee7-7413-4d4e-a711-b0cc9cdbbb29 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.498285 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c2ee7-7413-4d4e-a711-b0cc9cdbbb29 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.498560 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c2ee7-7413-4d4e-a711-b0cc9cdbbb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.516720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-0b7c2ee7-7413-4d4e-a711-b0cc9cdbbb29 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:49.517911 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0b7c2ee7-7413-4d4e-a711-b0cc9cdbbb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:49.535100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266b0c7-46e2-4c70-a5e9-fd4e7a959367 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:49.569662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4299e8b-0594-4d5c-850c-613e677cbbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:49.570262 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4299e8b-0594-4d5c-850c-613e677cbbc2 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.570548 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4299e8b-0594-4d5c-850c-613e677cbbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.570822 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4299e8b-0594-4d5c-850c-613e677cbbc2 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.571479 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4299e8b-0594-4d5c-850c-613e677cbbc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:49.571785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4299e8b-0594-4d5c-850c-613e677cbbc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:49.573188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4299e8b-0594-4d5c-850c-613e677cbbc2 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:49.580862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 292/569] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:49.581631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4299e8b-0594-4d5c-850c-613e677cbbc2 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:49.587674 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 278/570] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:49.587674 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c2ee7-7413-4d4e-a711-b0cc9cdbbb29 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:49.644085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:49.665555 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:49.682539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4299e8b-0594-4d5c-850c-613e677cbbc2 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:49.699749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c2ee7-7413-4d4e-a711-b0cc9cdbbb29 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:49.720684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe94e6-f727-4656-a17c-1649755dfbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:49.721316 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe94e6-f727-4656-a17c-1649755dfbf9 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.721591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe94e6-f727-4656-a17c-1649755dfbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.721862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe94e6-f727-4656-a17c-1649755dfbf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.729845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe94e6-f727-4656-a17c-1649755dfbf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:49.731390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe94e6-f727-4656-a17c-1649755dfbf9 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:49.736015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71725fbf-6904-45b7-a1e9-b6fdea7fb16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:49.736881 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71725fbf-6904-45b7-a1e9-b6fdea7fb16b None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.737337 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71725fbf-6904-45b7-a1e9-b6fdea7fb16b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.737780 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71725fbf-6904-45b7-a1e9-b6fdea7fb16b None admin] PATH_INFO: `/v3/users/nova` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-9abe94e6-f727-4656-a17c-1649755dfbf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:32:49.742714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:49.743552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:49.744454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:32:49.744454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:32:49.744454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:32:49.744454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:32:49.744454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:32:49.744454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:32:49.744454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:49.744454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:49.744454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 293/571] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:49.744961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe94e6-f727-4656-a17c-1649755dfbf9 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:49.747948 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71725fbf-6904-45b7-a1e9-b6fdea7fb16b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:49.749508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71725fbf-6904-45b7-a1e9-b6fdea7fb16b None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-71725fbf-6904-45b7-a1e9-b6fdea7fb16b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:32:49.762683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:49.763689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:49.764632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:49.764632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:49.764632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:32:49.764632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:32:49.764632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:32:49.764632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:32:49.764632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:32:49.764632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:32:49.764632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:32:49.764632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:49.764632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 279/572] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 24 07:32:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:49.764632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71725fbf-6904-45b7-a1e9-b6fdea7fb16b None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:49.855631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe94e6-f727-4656-a17c-1649755dfbf9 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:49.870060 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71725fbf-6904-45b7-a1e9-b6fdea7fb16b None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:49.878899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:49.890486 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afe0ec-51a6-45eb-884b-72325e70a59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:49.891113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afe0ec-51a6-45eb-884b-72325e70a59d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.891393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afe0ec-51a6-45eb-884b-72325e70a59d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.891667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afe0ec-51a6-45eb-884b-72325e70a59d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.899032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afe0ec-51a6-45eb-884b-72325e70a59d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:49.900456 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afe0ec-51a6-45eb-884b-72325e70a59d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:49.903906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71221c22-2bcd-44b9-894b-3f73a1825655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:49.904538 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71221c22-2bcd-44b9-894b-3f73a1825655 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:49.904846 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71221c22-2bcd-44b9-894b-3f73a1825655 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:49.905122 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71221c22-2bcd-44b9-894b-3f73a1825655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-d5afe0ec-51a6-45eb-884b-72325e70a59d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:32:49.911231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:49.912333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:49.913428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:49.913428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:49.913428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:32:49.913428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:32:49.913428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:32:49.913428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:32:49.913428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:32:49.913428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:32:49.913428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:32:49.913428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:49.916386 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71221c22-2bcd-44b9-894b-3f73a1825655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:49.916996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 294/573] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:49.917633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afe0ec-51a6-45eb-884b-72325e70a59d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:49.918026 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71221c22-2bcd-44b9-894b-3f73a1825655 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:49.921047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-71221c22-2bcd-44b9-894b-3f73a1825655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:49.932840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:49.933930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:49.934813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:49.934813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:49.934813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:49.934813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 280/574] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:49.935037 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71221c22-2bcd-44b9-894b-3f73a1825655 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:50.020620 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afe0ec-51a6-45eb-884b-72325e70a59d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:50.033989 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71221c22-2bcd-44b9-894b-3f73a1825655 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:50.054286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6c187-46a4-4d73-acd7-600e35169fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:50.054886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6c187-46a4-4d73-acd7-600e35169fe1 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:50.055167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6c187-46a4-4d73-acd7-600e35169fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:50.055438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6c187-46a4-4d73-acd7-600e35169fe1 None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:50.056118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6c187-46a4-4d73-acd7-600e35169fe1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:50.056422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6c187-46a4-4d73-acd7-600e35169fe1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:50.057885 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6c187-46a4-4d73-acd7-600e35169fe1 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:50.067442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807c8d7-9dcb-4fcd-bc12-f980c5e0fbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:50.068039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807c8d7-9dcb-4fcd-bc12-f980c5e0fbc2 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:50.068321 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807c8d7-9dcb-4fcd-bc12-f980c5e0fbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:50.068596 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807c8d7-9dcb-4fcd-bc12-f980c5e0fbc2 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:50.069349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807c8d7-9dcb-4fcd-bc12-f980c5e0fbc2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:50.069654 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807c8d7-9dcb-4fcd-bc12-f980c5e0fbc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:50.070343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 295/575] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:32:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:50.071065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807c8d7-9dcb-4fcd-bc12-f980c5e0fbc2 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:50.071173 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6c187-46a4-4d73-acd7-600e35169fe1 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:50.088177 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 281/576] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 24 07:32:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:50.089023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807c8d7-9dcb-4fcd-bc12-f980c5e0fbc2 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:50.108474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:50.135821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:50.172310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6c187-46a4-4d73-acd7-600e35169fe1 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:50.188464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807c8d7-9dcb-4fcd-bc12-f980c5e0fbc2 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:50.205698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cb725-6178-473b-8001-1d8976ca2a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:50.206319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb725-6178-473b-8001-1d8976ca2a26 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:50.206597 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb725-6178-473b-8001-1d8976ca2a26 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:50.206866 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb725-6178-473b-8001-1d8976ca2a26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:50.217056 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100cb725-6178-473b-8001-1d8976ca2a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:50.221754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f644498-ef9a-48c6-b1e1-2d224a2b8f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-100cb725-6178-473b-8001-1d8976ca2a26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:50.221964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:50.223725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:50.224673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:50.224673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:50.224673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:50.224673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:50.224673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:50.224673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:50.224673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:50.224673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:50.224673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:50.224673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:50.224673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:50.224673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:50.225323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f644498-ef9a-48c6-b1e1-2d224a2b8f41 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:50.225323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f644498-ef9a-48c6-b1e1-2d224a2b8f41 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:50.225323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f644498-ef9a-48c6-b1e1-2d224a2b8f41 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:50.225501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 296/577] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:50.225569 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f644498-ef9a-48c6-b1e1-2d224a2b8f41 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:50.225649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cb725-6178-473b-8001-1d8976ca2a26 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:50.225721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f644498-ef9a-48c6-b1e1-2d224a2b8f41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:50.226337 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f644498-ef9a-48c6-b1e1-2d224a2b8f41 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:50.243746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 282/578] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Aug 24 07:32:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:50.244551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f644498-ef9a-48c6-b1e1-2d224a2b8f41 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:50.319443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:50.323405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cb725-6178-473b-8001-1d8976ca2a26 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:50.341839 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f644498-ef9a-48c6-b1e1-2d224a2b8f41 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:50.346716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:50.357679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2445676-c01f-4e9e-846c-a0273586ced3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:50.358276 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2445676-c01f-4e9e-846c-a0273586ced3 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:50.358558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2445676-c01f-4e9e-846c-a0273586ced3 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:50.358833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2445676-c01f-4e9e-846c-a0273586ced3 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:50.359473 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2445676-c01f-4e9e-846c-a0273586ced3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:50.359874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2445676-c01f-4e9e-846c-a0273586ced3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:50.361303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2445676-c01f-4e9e-846c-a0273586ced3 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:50.372175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 297/579] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:50.372989 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2445676-c01f-4e9e-846c-a0273586ced3 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:50.375611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d62b53-295d-4263-8b8e-919dd60dabc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:50.376213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d62b53-295d-4263-8b8e-919dd60dabc4 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:50.376736 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d62b53-295d-4263-8b8e-919dd60dabc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:50.376896 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d62b53-295d-4263-8b8e-919dd60dabc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:50.377550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d62b53-295d-4263-8b8e-919dd60dabc4 None admin] RBAC: Adding query filter params (role.id=3acdf08298c84325b53a0e286cc53b30, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=b629ad3e6b9f4723b103f9845df2aaaf) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:50.377859 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d62b53-295d-4263-8b8e-919dd60dabc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:50.379988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d62b53-295d-4263-8b8e-919dd60dabc4 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:50.396683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 283/580] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:32:50 2022] GET /identity/v3/role_assignments?user.id=b629ad3e6b9f4723b103f9845df2aaaf&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=3acdf08298c84325b53a0e286cc53b30 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:50.396683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d62b53-295d-4263-8b8e-919dd60dabc4 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:50.492746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2445676-c01f-4e9e-846c-a0273586ced3 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:50.503571 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d62b53-295d-4263-8b8e-919dd60dabc4 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:50.531740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b586f7c-60d3-4d13-a318-78ec2d9bf66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:50.532155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b586f7c-60d3-4d13-a318-78ec2d9bf66d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:50.532438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b586f7c-60d3-4d13-a318-78ec2d9bf66d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:50.532746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b586f7c-60d3-4d13-a318-78ec2d9bf66d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:50.544336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b586f7c-60d3-4d13-a318-78ec2d9bf66d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:50.545823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b586f7c-60d3-4d13-a318-78ec2d9bf66d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:50.546699 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931e20c-0d69-4a8b-8df8-951b6de20c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:50.547312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931e20c-0d69-4a8b-8df8-951b6de20c3d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:50.547604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931e20c-0d69-4a8b-8df8-951b6de20c3d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:50.547885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931e20c-0d69-4a8b-8df8-951b6de20c3d None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:50.548553 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931e20c-0d69-4a8b-8df8-951b6de20c3d None admin] RBAC: Adding query filter params (name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:50.548961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931e20c-0d69-4a8b-8df8-951b6de20c3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:50.550456 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931e20c-0d69-4a8b-8df8-951b6de20c3d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:50.559179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 284/581] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:50.559808 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931e20c-0d69-4a8b-8df8-951b6de20c3d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-5b586f7c-60d3-4d13-a318-78ec2d9bf66d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:50.565882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:50.567535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:50.568441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:50.568441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:50.568441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:50.572242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 298/582] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:50.573048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b586f7c-60d3-4d13-a318-78ec2d9bf66d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:50.573562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:50.613197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:50.668991 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931e20c-0d69-4a8b-8df8-951b6de20c3d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:50.689518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b586f7c-60d3-4d13-a318-78ec2d9bf66d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:50.705806 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9617809-25e3-43a3-a646-13379d90e8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:50.706437 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9617809-25e3-43a3-a646-13379d90e8e9 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:50.706725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9617809-25e3-43a3-a646-13379d90e8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:50.707001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9617809-25e3-43a3-a646-13379d90e8e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:50.718211 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9617809-25e3-43a3-a646-13379d90e8e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:50.719742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9617809-25e3-43a3-a646-13379d90e8e9 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:50.726108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd6dfa-ac07-45d7-a553-eb476b7e512a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:50.727388 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd6dfa-ac07-45d7-a553-eb476b7e512a None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:50.727854 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd6dfa-ac07-45d7-a553-eb476b7e512a None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:50.728727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd6dfa-ac07-45d7-a553-eb476b7e512a None admin] PATH_INFO: `/v3/users/glance` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:50.736235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd6dfa-ac07-45d7-a553-eb476b7e512a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-e9617809-25e3-43a3-a646-13379d90e8e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:50.737237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:50.738180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:50.739614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:50.739614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:50.739614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:50.739614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 285/583] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:50.739614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9617809-25e3-43a3-a646-13379d90e8e9 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:50.740921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd6dfa-ac07-45d7-a553-eb476b7e512a None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-dfbd6dfa-ac07-45d7-a553-eb476b7e512a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:32:50.757796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:50.758767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:50.759725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:50.759725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:50.759725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:32:50.759725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:32:50.759725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:32:50.759725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:32:50.759725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:32:50.759725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:32:50.759725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:32:50.759725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:50.759725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 299/584] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:50.759725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd6dfa-ac07-45d7-a553-eb476b7e512a None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:50.804502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:50.854825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9617809-25e3-43a3-a646-13379d90e8e9 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:50.859466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:50.866895 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd6dfa-ac07-45d7-a553-eb476b7e512a None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:50.890013 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b43bbf-7bce-4c34-8520-957337a6d8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:50.890872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b43bbf-7bce-4c34-8520-957337a6d8cc None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:50.890982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b43bbf-7bce-4c34-8520-957337a6d8cc None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:50.891212 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b43bbf-7bce-4c34-8520-957337a6d8cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:50.902573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbdc35-be3f-48f4-b3fa-6e828f6c5714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:50.903054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b43bbf-7bce-4c34-8520-957337a6d8cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:50.903259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbdc35-be3f-48f4-b3fa-6e828f6c5714 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:50.903454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbdc35-be3f-48f4-b3fa-6e828f6c5714 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:50.903724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbdc35-be3f-48f4-b3fa-6e828f6c5714 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:50.904170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b43bbf-7bce-4c34-8520-957337a6d8cc None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:50.904364 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dbdc35-be3f-48f4-b3fa-6e828f6c5714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:50.904686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dbdc35-be3f-48f4-b3fa-6e828f6c5714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:50.906028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dbdc35-be3f-48f4-b3fa-6e828f6c5714 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:50.922621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 300/585] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:50.922621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbdc35-be3f-48f4-b3fa-6e828f6c5714 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-f4b43bbf-7bce-4c34-8520-957337a6d8cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:50.922885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:50.923724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:50.924602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:50.924602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:50.924602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:50.924602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 286/586] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:50.924602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b43bbf-7bce-4c34-8520-957337a6d8cc None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:51.026084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b43bbf-7bce-4c34-8520-957337a6d8cc None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:51.026263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbdc35-be3f-48f4-b3fa-6e828f6c5714 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:51.032971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:51.059413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696672f9-3948-488a-85d9-56a56beac052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:51.059732 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c4b11-99b6-4cfa-9c61-9eda7ccfa6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:51.060019 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696672f9-3948-488a-85d9-56a56beac052 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:51.060298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696672f9-3948-488a-85d9-56a56beac052 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:51.060396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c4b11-99b6-4cfa-9c61-9eda7ccfa6ca None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:51.060613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696672f9-3948-488a-85d9-56a56beac052 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:51.060766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c4b11-99b6-4cfa-9c61-9eda7ccfa6ca None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:51.060920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c4b11-99b6-4cfa-9c61-9eda7ccfa6ca None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:51.061246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696672f9-3948-488a-85d9-56a56beac052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:51.061551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696672f9-3948-488a-85d9-56a56beac052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:51.061649 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c4b11-99b6-4cfa-9c61-9eda7ccfa6ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:51.061930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c4b11-99b6-4cfa-9c61-9eda7ccfa6ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:51.062912 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696672f9-3948-488a-85d9-56a56beac052 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:51.063338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c4b11-99b6-4cfa-9c61-9eda7ccfa6ca None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:51.079346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 301/587] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:51.079346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696672f9-3948-488a-85d9-56a56beac052 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:51.084305 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 287/588] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Aug 24 07:32:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:51.088781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c4b11-99b6-4cfa-9c61-9eda7ccfa6ca None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:51.099093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:51.181651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696672f9-3948-488a-85d9-56a56beac052 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:51.194702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c4b11-99b6-4cfa-9c61-9eda7ccfa6ca None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:51.218808 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f0fe1-d1ea-4911-a19c-a294d1c1f36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:51.219777 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f0fe1-d1ea-4911-a19c-a294d1c1f36b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:51.220058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f0fe1-d1ea-4911-a19c-a294d1c1f36b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:51.220330 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f0fe1-d1ea-4911-a19c-a294d1c1f36b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:51.227828 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f0fe1-d1ea-4911-a19c-a294d1c1f36b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:51.229307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f0fe1-d1ea-4911-a19c-a294d1c1f36b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:51.231189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77348f-8279-4539-9ce4-b5505499b417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:51.231910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77348f-8279-4539-9ce4-b5505499b417 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:51.232196 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77348f-8279-4539-9ce4-b5505499b417 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:51.232473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77348f-8279-4539-9ce4-b5505499b417 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:51.233146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77348f-8279-4539-9ce4-b5505499b417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:51.233452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77348f-8279-4539-9ce4-b5505499b417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:51.234827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77348f-8279-4539-9ce4-b5505499b417 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-9f5f0fe1-d1ea-4911-a19c-a294d1c1f36b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:32:51.241293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:51.242230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:51.243099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:32:51.243099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:32:51.243099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:32:51.243099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:32:51.243099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:32:51.243099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:32:51.243099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:51.243099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:51.243099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 302/589] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:51.244591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f0fe1-d1ea-4911-a19c-a294d1c1f36b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:51.251914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 288/590] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:51.252601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77348f-8279-4539-9ce4-b5505499b417 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:51.279346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:51.334571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:51.357680 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f0fe1-d1ea-4911-a19c-a294d1c1f36b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:51.358572 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77348f-8279-4539-9ce4-b5505499b417 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:51.395045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4e072-30d0-4d38-a628-e42b1d22d5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:51.395510 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f416f3-8e10-4b09-9ca1-e49384f94ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:51.395816 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4e072-30d0-4d38-a628-e42b1d22d5d7 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:51.396103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4e072-30d0-4d38-a628-e42b1d22d5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:51.396224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f416f3-8e10-4b09-9ca1-e49384f94ee4 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:51.396401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f416f3-8e10-4b09-9ca1-e49384f94ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:51.396494 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4e072-30d0-4d38-a628-e42b1d22d5d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:51.396711 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f416f3-8e10-4b09-9ca1-e49384f94ee4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:51.403297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4e072-30d0-4d38-a628-e42b1d22d5d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:51.405017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4e072-30d0-4d38-a628-e42b1d22d5d7 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:51.407310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f416f3-8e10-4b09-9ca1-e49384f94ee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:51.408812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f416f3-8e10-4b09-9ca1-e49384f94ee4 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-70e4e072-30d0-4d38-a628-e42b1d22d5d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:32:51.417080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:51.417945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:51.418833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:32:51.418833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:32:51.418833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:32:51.418833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:32:51.418833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:32:51.418833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:32:51.418833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:51.418833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:51.418833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 303/591] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:51.419889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4e072-30d0-4d38-a628-e42b1d22d5d7 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-75f416f3-8e10-4b09-9ca1-e49384f94ee4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:51.426578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:51.427520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:51.428425 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:51.428425 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:51.428425 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:51.428425 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 289/592] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:51.428665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f416f3-8e10-4b09-9ca1-e49384f94ee4 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:51.510002 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:51.532259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f416f3-8e10-4b09-9ca1-e49384f94ee4 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:51.540116 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4e072-30d0-4d38-a628-e42b1d22d5d7 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:51.567558 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dedd0b4-ef0e-46fd-936b-a96a46b27aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:51.568286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedd0b4-ef0e-46fd-936b-a96a46b27aa1 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:51.568626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedd0b4-ef0e-46fd-936b-a96a46b27aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:51.568998 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedd0b4-ef0e-46fd-936b-a96a46b27aa1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:51.576569 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dedd0b4-ef0e-46fd-936b-a96a46b27aa1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:51.578056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dedd0b4-ef0e-46fd-936b-a96a46b27aa1 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:51.578888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f057-b4cf-491f-8df1-b62457e6fc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:51.579610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f057-b4cf-491f-8df1-b62457e6fc7e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:51.579843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f057-b4cf-491f-8df1-b62457e6fc7e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:51.580122 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f057-b4cf-491f-8df1-b62457e6fc7e None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:51.581049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48f057-b4cf-491f-8df1-b62457e6fc7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:51.581222 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48f057-b4cf-491f-8df1-b62457e6fc7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:51.583143 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48f057-b4cf-491f-8df1-b62457e6fc7e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:51.586036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-7dedd0b4-ef0e-46fd-936b-a96a46b27aa1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:32:51.590384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:51.591320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:51.592371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:51.592371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:51.592371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:32:51.592371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:32:51.592371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:32:51.592371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:32:51.592371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:32:51.592371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:32:51.592371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:32:51.592371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:51.592371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 290/593] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:51.592371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dedd0b4-ef0e-46fd-936b-a96a46b27aa1 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:51.606403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 304/594] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:32:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:51.606403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f057-b4cf-491f-8df1-b62457e6fc7e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:51.701461 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dedd0b4-ef0e-46fd-936b-a96a46b27aa1 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:51.711609 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f057-b4cf-491f-8df1-b62457e6fc7e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:51.738158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189aee30-5547-41c2-95b4-278edd72bcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:51.739108 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189aee30-5547-41c2-95b4-278edd72bcbd None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:51.739553 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189aee30-5547-41c2-95b4-278edd72bcbd None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:51.739829 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189aee30-5547-41c2-95b4-278edd72bcbd None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:51.740520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189aee30-5547-41c2-95b4-278edd72bcbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:51.740861 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189aee30-5547-41c2-95b4-278edd72bcbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:51.742283 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189aee30-5547-41c2-95b4-278edd72bcbd None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:51.745171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:51.747509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef748b-b22b-437e-8525-11863588c639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:51.748114 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef748b-b22b-437e-8525-11863588c639 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:51.748397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef748b-b22b-437e-8525-11863588c639 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:51.748693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef748b-b22b-437e-8525-11863588c639 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:51.749334 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef748b-b22b-437e-8525-11863588c639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:51.749641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef748b-b22b-437e-8525-11863588c639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:51.750998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef748b-b22b-437e-8525-11863588c639 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:51.755614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 291/595] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:32:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:51.756334 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189aee30-5547-41c2-95b4-278edd72bcbd None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:51.773435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 305/596] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:51.773435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef748b-b22b-437e-8525-11863588c639 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:51.828790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:51.867172 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189aee30-5547-41c2-95b4-278edd72bcbd None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:51.878001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef748b-b22b-437e-8525-11863588c639 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:51.903492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68a95e-4b52-47de-ba23-249ec600c111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:51.904098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68a95e-4b52-47de-ba23-249ec600c111 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:51.904377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68a95e-4b52-47de-ba23-249ec600c111 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:51.904744 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68a95e-4b52-47de-ba23-249ec600c111 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:51.905395 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68a95e-4b52-47de-ba23-249ec600c111 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:51.905698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68a95e-4b52-47de-ba23-249ec600c111 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:51.907237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68a95e-4b52-47de-ba23-249ec600c111 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:51.914902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bce0c-700d-421a-9ad9-4512d6d18ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:51.915570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bce0c-700d-421a-9ad9-4512d6d18ee9 None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:51.915857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bce0c-700d-421a-9ad9-4512d6d18ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:51.916137 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bce0c-700d-421a-9ad9-4512d6d18ee9 None admin] PATH_INFO: `/v3/projects/fc060764023e44a7ade80eeed98a235f/users/bdf81f09006a42299c0807f77f16c748/roles/3acdf08298c84325b53a0e286cc53b30` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:51.929789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 292/597] 10.222.0.26 () {56 vars in 1139 bytes} [Wed Aug 24 07:32:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:51.929789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68a95e-4b52-47de-ba23-249ec600c111 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:51.935622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bce0c-700d-421a-9ad9-4512d6d18ee9 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc060764023e44a7ade80eeed98a235f, user_id=bdf81f09006a42299c0807f77f16c748, role_id=3acdf08298c84325b53a0e286cc53b30)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:51.937176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bce0c-700d-421a-9ad9-4512d6d18ee9 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:51.983817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:52.016591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 306/598] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:32:51 2022] PUT /identity/v3/projects/fc060764023e44a7ade80eeed98a235f/users/bdf81f09006a42299c0807f77f16c748/roles/3acdf08298c84325b53a0e286cc53b30 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:52.016591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bce0c-700d-421a-9ad9-4512d6d18ee9 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:52.040544 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68a95e-4b52-47de-ba23-249ec600c111 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:52.083799 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6f8e2-a15e-425c-bdc8-4531b14ae70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:52.084418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6f8e2-a15e-425c-bdc8-4531b14ae70d None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:52.084741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6f8e2-a15e-425c-bdc8-4531b14ae70d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:52.085024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6f8e2-a15e-425c-bdc8-4531b14ae70d None admin] PATH_INFO: `/v3/projects/fc060764023e44a7ade80eeed98a235f/users/43e57131a5be4af785166bc03a3337a2/roles/3acdf08298c84325b53a0e286cc53b30` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:52.103028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6f8e2-a15e-425c-bdc8-4531b14ae70d None admin] RBAC: Authorizing `identity:create_grant(project_id=fc060764023e44a7ade80eeed98a235f, user_id=43e57131a5be4af785166bc03a3337a2, role_id=3acdf08298c84325b53a0e286cc53b30)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:52.104549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6f8e2-a15e-425c-bdc8-4531b14ae70d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:52.156623 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bce0c-700d-421a-9ad9-4512d6d18ee9 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:52.167943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:52.175736 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 293/599] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:32:51 2022] PUT /identity/v3/projects/fc060764023e44a7ade80eeed98a235f/users/43e57131a5be4af785166bc03a3337a2/roles/3acdf08298c84325b53a0e286cc53b30 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:52.177669 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6f8e2-a15e-425c-bdc8-4531b14ae70d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:52.203630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7589dc-2fc5-4b23-aca6-ad8a1722b0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:52.204288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7589dc-2fc5-4b23-aca6-ad8a1722b0ec None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:52.204568 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7589dc-2fc5-4b23-aca6-ad8a1722b0ec None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:52.204861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7589dc-2fc5-4b23-aca6-ad8a1722b0ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:52.212843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7589dc-2fc5-4b23-aca6-ad8a1722b0ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:52.227937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7589dc-2fc5-4b23-aca6-ad8a1722b0ec None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:52.227937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-4c7589dc-2fc5-4b23-aca6-ad8a1722b0ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:52.227937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:52.227937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:52.227937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:52.227937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:52.227937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:52.227937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:52.227937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:52.227937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:52.227937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:52.227937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:52.227937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:52.227937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:52.228803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:52.229700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:52.229700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:52.229700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:32:52.229700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:32:52.229700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:32:52.229700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:32:52.229700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:32:52.229700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:32:52.229700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:52.229700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:52.229700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 307/600] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:52.235976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7589dc-2fc5-4b23-aca6-ad8a1722b0ec None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:52.334965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6f8e2-a15e-425c-bdc8-4531b14ae70d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:52.339643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:52.383989 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5616ae-6c3b-4951-ab64-9205af56c7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:52.384988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5616ae-6c3b-4951-ab64-9205af56c7f9 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:52.385369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5616ae-6c3b-4951-ab64-9205af56c7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:52.385747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5616ae-6c3b-4951-ab64-9205af56c7f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:52.397600 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7589dc-2fc5-4b23-aca6-ad8a1722b0ec None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:52.398828 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5616ae-6c3b-4951-ab64-9205af56c7f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:52.400349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5616ae-6c3b-4951-ab64-9205af56c7f9 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-dc5616ae-6c3b-4951-ab64-9205af56c7f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:52.417906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:52.419021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:52.420093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:52.420093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:52.420093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:52.421616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 294/601] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:52.423066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634d8da-fdfa-41ad-9e03-42d3ed57788c None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:52.423517 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634d8da-fdfa-41ad-9e03-42d3ed57788c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:52.424036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634d8da-fdfa-41ad-9e03-42d3ed57788c None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:52.426135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 295/602] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:52.427114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634d8da-fdfa-41ad-9e03-42d3ed57788c None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:52.446666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54af16c-9fbb-4a7b-aead-00ffa1cbd3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:52.447868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54af16c-9fbb-4a7b-aead-00ffa1cbd3e2 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:52.448394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54af16c-9fbb-4a7b-aead-00ffa1cbd3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:52.448913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54af16c-9fbb-4a7b-aead-00ffa1cbd3e2 None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:52.449872 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54af16c-9fbb-4a7b-aead-00ffa1cbd3e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:52.450492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54af16c-9fbb-4a7b-aead-00ffa1cbd3e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:52.452214 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54af16c-9fbb-4a7b-aead-00ffa1cbd3e2 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:52.467430 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 308/603] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:32:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:52.469217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70ee93-2ea3-4fd6-9a15-3d6eb0afbc99 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:52.469732 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70ee93-2ea3-4fd6-9a15-3d6eb0afbc99 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:52.470197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70ee93-2ea3-4fd6-9a15-3d6eb0afbc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:52.490147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-ef70ee93-2ea3-4fd6-9a15-3d6eb0afbc99 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:52.491807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ef70ee93-2ea3-4fd6-9a15-3d6eb0afbc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:52.515275 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:52.546911 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634d8da-fdfa-41ad-9e03-42d3ed57788c None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:52.583610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 309/604] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:52.584359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70ee93-2ea3-4fd6-9a15-3d6eb0afbc99 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:52.589551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e06c1b-5421-4eee-a79c-30881c6514db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:52.590128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e06c1b-5421-4eee-a79c-30881c6514db None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:52.590412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e06c1b-5421-4eee-a79c-30881c6514db None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:52.590689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e06c1b-5421-4eee-a79c-30881c6514db None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:52.591347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e06c1b-5421-4eee-a79c-30881c6514db None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:52.591652 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e06c1b-5421-4eee-a79c-30881c6514db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:52.598731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e06c1b-5421-4eee-a79c-30881c6514db None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:52.620384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 296/605] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:52.630197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e06c1b-5421-4eee-a79c-30881c6514db None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:52.657273 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:52.692578 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70ee93-2ea3-4fd6-9a15-3d6eb0afbc99 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:52.729686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb382ce-e98e-4b7f-b286-2d972ef1415f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:52.730392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb382ce-e98e-4b7f-b286-2d972ef1415f None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:52.730691 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb382ce-e98e-4b7f-b286-2d972ef1415f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:52.730965 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb382ce-e98e-4b7f-b286-2d972ef1415f None admin] PATH_INFO: `/v3/projects/fc060764023e44a7ade80eeed98a235f/users/38de805ba5eb4aa58cf95d2082c2d216/roles/3acdf08298c84325b53a0e286cc53b30` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:52.750497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb382ce-e98e-4b7f-b286-2d972ef1415f None admin] RBAC: Authorizing `identity:create_grant(project_id=fc060764023e44a7ade80eeed98a235f, user_id=38de805ba5eb4aa58cf95d2082c2d216, role_id=3acdf08298c84325b53a0e286cc53b30)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:52.752661 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb382ce-e98e-4b7f-b286-2d972ef1415f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:52.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e06c1b-5421-4eee-a79c-30881c6514db None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:52.811583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265ec4e-ec1e-4cdd-b695-f37594356144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:52.812310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265ec4e-ec1e-4cdd-b695-f37594356144 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:52.812656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265ec4e-ec1e-4cdd-b695-f37594356144 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:52.813009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265ec4e-ec1e-4cdd-b695-f37594356144 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:52.820090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265ec4e-ec1e-4cdd-b695-f37594356144 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:52.821714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265ec4e-ec1e-4cdd-b695-f37594356144 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-e265ec4e-ec1e-4cdd-b695-f37594356144 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:32:52.835064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:52.835967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:52.836862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:32:52.836862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:32:52.836862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:32:52.836862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:32:52.836862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:32:52.836862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:32:52.836862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:52.836862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:52.836862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 297/606] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:52.842339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265ec4e-ec1e-4cdd-b695-f37594356144 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:52.854263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 310/607] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:32:52 2022] PUT /identity/v3/projects/fc060764023e44a7ade80eeed98a235f/users/38de805ba5eb4aa58cf95d2082c2d216/roles/3acdf08298c84325b53a0e286cc53b30 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:52.960673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:52.993473 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb382ce-e98e-4b7f-b286-2d972ef1415f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:52.994934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265ec4e-ec1e-4cdd-b695-f37594356144 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:53.003946 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:53.050065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde32d9-bd09-464e-864d-0fb455d6eeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:53.050680 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde32d9-bd09-464e-864d-0fb455d6eeb9 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:53.051547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde32d9-bd09-464e-864d-0fb455d6eeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:53.051821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde32d9-bd09-464e-864d-0fb455d6eeb9 None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:53.052512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde32d9-bd09-464e-864d-0fb455d6eeb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:53.052851 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde32d9-bd09-464e-864d-0fb455d6eeb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:53.054758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde32d9-bd09-464e-864d-0fb455d6eeb9 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:53.068526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 298/608] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:32:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:53.074628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde32d9-bd09-464e-864d-0fb455d6eeb9 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:53.190039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde32d9-bd09-464e-864d-0fb455d6eeb9 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:53.194257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb382ce-e98e-4b7f-b286-2d972ef1415f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:53.226014 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5f54f-0e88-4ca5-9533-2ce42ca80fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:53.226693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5f54f-0e88-4ca5-9533-2ce42ca80fe1 None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:53.226980 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5f54f-0e88-4ca5-9533-2ce42ca80fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:53.227260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5f54f-0e88-4ca5-9533-2ce42ca80fe1 None admin] PATH_INFO: `/v3/projects/fc060764023e44a7ade80eeed98a235f/users/46e61402c01c4f9ba5f18e94960bc313/roles/3acdf08298c84325b53a0e286cc53b30` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:53.231525 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:53.244431 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5f54f-0e88-4ca5-9533-2ce42ca80fe1 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc060764023e44a7ade80eeed98a235f, user_id=46e61402c01c4f9ba5f18e94960bc313, role_id=3acdf08298c84325b53a0e286cc53b30)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:53.249751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8147a-6c4c-442a-a264-a6fbb4bbb52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:53.249882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5f54f-0e88-4ca5-9533-2ce42ca80fe1 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:53.249970 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8147a-6c4c-442a-a264-a6fbb4bbb52d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:53.249970 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8147a-6c4c-442a-a264-a6fbb4bbb52d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:53.249970 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8147a-6c4c-442a-a264-a6fbb4bbb52d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:53.263586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8147a-6c4c-442a-a264-a6fbb4bbb52d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-72b8147a-6c4c-442a-a264-a6fbb4bbb52d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:53.269003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:53.273295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:53.274118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:53.274118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:53.274118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:53.274118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:53.274118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:53.274118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:53.274118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:53.274118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:53.274118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:53.274118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:53.274118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:53.274118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:53.274118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 311/609] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:32:53.277770 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8147a-6c4c-442a-a264-a6fbb4bbb52d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:53.322670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 299/610] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:32:53 2022] PUT /identity/v3/projects/fc060764023e44a7ade80eeed98a235f/users/46e61402c01c4f9ba5f18e94960bc313/roles/3acdf08298c84325b53a0e286cc53b30 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:53.388318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:53.431195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8147a-6c4c-442a-a264-a6fbb4bbb52d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:53.480925 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7666cde-8a57-4d40-ab5c-6090e573dc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:53.481459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7666cde-8a57-4d40-ab5c-6090e573dc3b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:53.481785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7666cde-8a57-4d40-ab5c-6090e573dc3b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:53.482062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7666cde-8a57-4d40-ab5c-6090e573dc3b None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:53.482727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7666cde-8a57-4d40-ab5c-6090e573dc3b None admin] RBAC: Adding query filter params (name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:53.483031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7666cde-8a57-4d40-ab5c-6090e573dc3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:53.484403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7666cde-8a57-4d40-ab5c-6090e573dc3b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:53.496286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 312/611] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:53.501963 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5f54f-0e88-4ca5-9533-2ce42ca80fe1 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:53.613738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5f54f-0e88-4ca5-9533-2ce42ca80fe1 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:53.649780 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c85901-edb9-4a62-8e80-dff4fb521ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:53.650457 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c85901-edb9-4a62-8e80-dff4fb521ce1 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:53.650739 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c85901-edb9-4a62-8e80-dff4fb521ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:53.651015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c85901-edb9-4a62-8e80-dff4fb521ce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:53.662363 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c85901-edb9-4a62-8e80-dff4fb521ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:53.663822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c85901-edb9-4a62-8e80-dff4fb521ce1 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-89c85901-edb9-4a62-8e80-dff4fb521ce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:53.679084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:53.680032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:53.680945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:53.680945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:53.680945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:53.680945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 300/612] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:53.686309 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7666cde-8a57-4d40-ab5c-6090e573dc3b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:53.812532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7666cde-8a57-4d40-ab5c-6090e573dc3b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:53.846684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:53.858295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f6c68-82ef-4449-a4d8-53baf06539f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:53.858911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f6c68-82ef-4449-a4d8-53baf06539f1 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:53.861779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f6c68-82ef-4449-a4d8-53baf06539f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:53.861779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f6c68-82ef-4449-a4d8-53baf06539f1 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:53.861779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f6c68-82ef-4449-a4d8-53baf06539f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:53.861779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f6c68-82ef-4449-a4d8-53baf06539f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:53.862120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f6c68-82ef-4449-a4d8-53baf06539f1 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:53.882679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 313/613] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:53.885362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c85901-edb9-4a62-8e80-dff4fb521ce1 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:53.956559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:54.008011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c85901-edb9-4a62-8e80-dff4fb521ce1 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:54.043967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c20b14-e3de-4c9a-9774-8f5be32e7281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:54.044603 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c20b14-e3de-4c9a-9774-8f5be32e7281 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:54.044976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c20b14-e3de-4c9a-9774-8f5be32e7281 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:54.045275 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c20b14-e3de-4c9a-9774-8f5be32e7281 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:54.055124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c20b14-e3de-4c9a-9774-8f5be32e7281 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:54.056698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c20b14-e3de-4c9a-9774-8f5be32e7281 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-55c20b14-e3de-4c9a-9774-8f5be32e7281 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:32:54.069355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:54.070304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:54.071862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:54.071862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:54.071862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:32:54.071862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:32:54.071862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:32:54.071862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:32:54.071862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:32:54.071862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:32:54.071862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:32:54.071862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:54.071862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 301/614] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:32:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:54.076381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f6c68-82ef-4449-a4d8-53baf06539f1 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:54.209836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f6c68-82ef-4449-a4d8-53baf06539f1 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:54.255328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48386dd4-2cd9-4b03-8a77-9536b9f782f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:54.255943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48386dd4-2cd9-4b03-8a77-9536b9f782f6 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:54.256220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48386dd4-2cd9-4b03-8a77-9536b9f782f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:54.256489 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48386dd4-2cd9-4b03-8a77-9536b9f782f6 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:54.257243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48386dd4-2cd9-4b03-8a77-9536b9f782f6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:54.257548 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48386dd4-2cd9-4b03-8a77-9536b9f782f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:54.258963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48386dd4-2cd9-4b03-8a77-9536b9f782f6 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:54.289444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 314/615] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Aug 24 07:32:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:54.295393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c20b14-e3de-4c9a-9774-8f5be32e7281 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:54.391943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6a710-3c62-4ba4-8914-573d1eabef7d None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:54.392256 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6a710-3c62-4ba4-8914-573d1eabef7d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:54.392424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6a710-3c62-4ba4-8914-573d1eabef7d None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:54.394304 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 315/616] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:54.401183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1660da-85fe-4353-a963-9a5ee5233435 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:54.401471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1660da-85fe-4353-a963-9a5ee5233435 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:54.401748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1660da-85fe-4353-a963-9a5ee5233435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:54.437700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-ab1660da-85fe-4353-a963-9a5ee5233435 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:54.438912 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ab1660da-85fe-4353-a963-9a5ee5233435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:54.445654 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c20b14-e3de-4c9a-9774-8f5be32e7281 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:54.448950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:54.484062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2ce52-8f6f-469b-9247-b776a50c9a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:54.484732 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2ce52-8f6f-469b-9247-b776a50c9a82 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:54.485017 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2ce52-8f6f-469b-9247-b776a50c9a82 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:54.485363 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2ce52-8f6f-469b-9247-b776a50c9a82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:54.497425 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2ce52-8f6f-469b-9247-b776a50c9a82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:54.499065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2ce52-8f6f-469b-9247-b776a50c9a82 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:54.502222 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-d3c2ce52-8f6f-469b-9247-b776a50c9a82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:54.515378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:54.521011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:54.521931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:54.521931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:54.521931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:54.521931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 302/617] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:54.522158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb7ee6-8359-4c69-8e15-f705ba17c3bc None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:54.524222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb7ee6-8359-4c69-8e15-f705ba17c3bc None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:54.524222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb7ee6-8359-4c69-8e15-f705ba17c3bc None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:54.524222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 303/618] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:54.525761 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb7ee6-8359-4c69-8e15-f705ba17c3bc None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:54.534253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 316/619] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:54.540694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501d6da-a6bd-4e7e-9a3a-38f86ea714d9 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:54.540694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501d6da-a6bd-4e7e-9a3a-38f86ea714d9 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:54.540694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501d6da-a6bd-4e7e-9a3a-38f86ea714d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:54.576472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a501d6da-a6bd-4e7e-9a3a-38f86ea714d9 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:54.577692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a501d6da-a6bd-4e7e-9a3a-38f86ea714d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:54.709455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb7ee6-8359-4c69-8e15-f705ba17c3bc None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:54.719113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 317/620] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:54.763192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e1403-d5b7-4345-bb59-b44d4340ad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:54.763801 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e1403-d5b7-4345-bb59-b44d4340ad88 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:54.764082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e1403-d5b7-4345-bb59-b44d4340ad88 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:54.764358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e1403-d5b7-4345-bb59-b44d4340ad88 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:54.765044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e1403-d5b7-4345-bb59-b44d4340ad88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:54.765349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e1403-d5b7-4345-bb59-b44d4340ad88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:54.766720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e1403-d5b7-4345-bb59-b44d4340ad88 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:54.779990 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 304/621] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:54.786359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501d6da-a6bd-4e7e-9a3a-38f86ea714d9 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:54.860866 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e1403-d5b7-4345-bb59-b44d4340ad88 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:54.881349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:54.965885 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501d6da-a6bd-4e7e-9a3a-38f86ea714d9 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:54.978286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:55.030501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6109b-e62f-4db8-a14f-689b602f74ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:55.031480 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6109b-e62f-4db8-a14f-689b602f74ae None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:55.033021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6109b-e62f-4db8-a14f-689b602f74ae None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:55.033502 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6109b-e62f-4db8-a14f-689b602f74ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:55.042737 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6109b-e62f-4db8-a14f-689b602f74ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:55.044531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6109b-e62f-4db8-a14f-689b602f74ae None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-1da6109b-e62f-4db8-a14f-689b602f74ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:32:55.057746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:55.058663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:55.059549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:32:55.059549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:32:55.059549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:32:55.059549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:32:55.059549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:32:55.059549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:32:55.059549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:55.059549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:55.062013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 318/622] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:55.066466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e1403-d5b7-4345-bb59-b44d4340ad88 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:55.068044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6109b-e62f-4db8-a14f-689b602f74ae None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:55.137934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba768180-f425-4992-a12b-7775b4be25f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:55.138561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba768180-f425-4992-a12b-7775b4be25f1 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:55.138837 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba768180-f425-4992-a12b-7775b4be25f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:55.139107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba768180-f425-4992-a12b-7775b4be25f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:55.152574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba768180-f425-4992-a12b-7775b4be25f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-ba768180-f425-4992-a12b-7775b4be25f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:55.158848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:55.159796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:55.160905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:55.160905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:55.160905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:55.160905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:55.160905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:55.160905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:55.160905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:55.160905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:55.160905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:55.160905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:55.160905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:55.160905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:55.160905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 305/623] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:32:55.162158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba768180-f425-4992-a12b-7775b4be25f1 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:55.217810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6109b-e62f-4db8-a14f-689b602f74ae None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:55.252521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:55.263828 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88ee8a-6656-4246-b759-e4cb48d0c686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:55.264432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88ee8a-6656-4246-b759-e4cb48d0c686 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:55.265911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88ee8a-6656-4246-b759-e4cb48d0c686 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:55.265911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88ee8a-6656-4246-b759-e4cb48d0c686 None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:55.265911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88ee8a-6656-4246-b759-e4cb48d0c686 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:55.266165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88ee8a-6656-4246-b759-e4cb48d0c686 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:55.267486 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88ee8a-6656-4246-b759-e4cb48d0c686 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:55.280341 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 319/624] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:32:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:32:55.281083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88ee8a-6656-4246-b759-e4cb48d0c686 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:55.292839 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba768180-f425-4992-a12b-7775b4be25f1 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:55.312715 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:55.328329 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a06774-285c-466c-be82-16aeafb7fb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:55.328992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a06774-285c-466c-be82-16aeafb7fb1f None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:55.329270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a06774-285c-466c-be82-16aeafb7fb1f None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:55.329563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a06774-285c-466c-be82-16aeafb7fb1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:55.341485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a06774-285c-466c-be82-16aeafb7fb1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-51a06774-285c-466c-be82-16aeafb7fb1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:55.346239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:55.347199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:55.348407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:55.348407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:55.348407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:55.348407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:55.348407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:55.348407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:55.348407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:55.348407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:55.348407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:55.348407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:55.348407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:55.348407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:55.348407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 306/625] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:55.349609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863dd18-0790-40b9-8bdf-f759cf5c8e2e None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:55.349609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863dd18-0790-40b9-8bdf-f759cf5c8e2e None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:55.349609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863dd18-0790-40b9-8bdf-f759cf5c8e2e None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:55.350932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 307/626] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:55.353678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863dd18-0790-40b9-8bdf-f759cf5c8e2e None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:55.395910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88ee8a-6656-4246-b759-e4cb48d0c686 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:55.431593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da071a-576b-4018-a319-6228fbbba0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:55.432207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da071a-576b-4018-a319-6228fbbba0eb None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:55.432488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da071a-576b-4018-a319-6228fbbba0eb None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:55.432821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da071a-576b-4018-a319-6228fbbba0eb None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:55.433483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da071a-576b-4018-a319-6228fbbba0eb None admin] RBAC: Adding query filter params (name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:55.433793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da071a-576b-4018-a319-6228fbbba0eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:55.435166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da071a-576b-4018-a319-6228fbbba0eb None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:55.444170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 320/627] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:55.445228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da071a-576b-4018-a319-6228fbbba0eb None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:55.473312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863dd18-0790-40b9-8bdf-f759cf5c8e2e None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:55.503535 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:55.510742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51b963-d744-4f1f-a8c4-92e7b757c48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:55.511580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51b963-d744-4f1f-a8c4-92e7b757c48f None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:55.511993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51b963-d744-4f1f-a8c4-92e7b757c48f None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:55.512452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51b963-d744-4f1f-a8c4-92e7b757c48f None admin] PATH_INFO: `/v3/projects/fc060764023e44a7ade80eeed98a235f/users/b629ad3e6b9f4723b103f9845df2aaaf/roles/3acdf08298c84325b53a0e286cc53b30` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:55.530657 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51b963-d744-4f1f-a8c4-92e7b757c48f None admin] RBAC: Authorizing `identity:create_grant(project_id=fc060764023e44a7ade80eeed98a235f, user_id=b629ad3e6b9f4723b103f9845df2aaaf, role_id=3acdf08298c84325b53a0e286cc53b30)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:55.532115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51b963-d744-4f1f-a8c4-92e7b757c48f None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:55.537643 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:55.549377 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da071a-576b-4018-a319-6228fbbba0eb None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:55.583018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b828cd8-4ff7-4fe5-ae77-15633996811e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:55.583622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b828cd8-4ff7-4fe5-ae77-15633996811e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:55.583905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b828cd8-4ff7-4fe5-ae77-15633996811e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:55.584180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b828cd8-4ff7-4fe5-ae77-15633996811e None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:55.584871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b828cd8-4ff7-4fe5-ae77-15633996811e None admin] RBAC: Adding query filter params (name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:55.585182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b828cd8-4ff7-4fe5-ae77-15633996811e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:55.586532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b828cd8-4ff7-4fe5-ae77-15633996811e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:55.602098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 321/628] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:55.602098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096596a-636b-42b2-beb1-08884c6ad8c9 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:55.602098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096596a-636b-42b2-beb1-08884c6ad8c9 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:55.602098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096596a-636b-42b2-beb1-08884c6ad8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:55.608415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 308/629] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:32:55 2022] PUT /identity/v3/projects/fc060764023e44a7ade80eeed98a235f/users/b629ad3e6b9f4723b103f9845df2aaaf/roles/3acdf08298c84325b53a0e286cc53b30 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:32:55.615969 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51b963-d744-4f1f-a8c4-92e7b757c48f None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:55.619609 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-e096596a-636b-42b2-beb1-08884c6ad8c9 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:55.621122 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e096596a-636b-42b2-beb1-08884c6ad8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:55.709067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 322/630] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:55.710270 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096596a-636b-42b2-beb1-08884c6ad8c9 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:55.769159 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51b963-d744-4f1f-a8c4-92e7b757c48f None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:55.816183 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b480076-7250-4f47-83ea-4696933b7961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:55.817086 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b480076-7250-4f47-83ea-4696933b7961 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:55.817294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b480076-7250-4f47-83ea-4696933b7961 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:55.817567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b480076-7250-4f47-83ea-4696933b7961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:55.832978 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:55.834822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b480076-7250-4f47-83ea-4696933b7961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:55.836985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b480076-7250-4f47-83ea-4696933b7961 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-3b480076-7250-4f47-83ea-4696933b7961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:55.853223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:55.854405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:55.855615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:55.855615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:55.855615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:55.855615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 309/631] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:55.855876 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df0d3c-8e6e-4fca-98f7-1b1b4c7542c5 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:55.856104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df0d3c-8e6e-4fca-98f7-1b1b4c7542c5 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:55.856385 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df0d3c-8e6e-4fca-98f7-1b1b4c7542c5 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:55.857932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 310/632] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:55.860033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096596a-636b-42b2-beb1-08884c6ad8c9 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:55.860721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df0d3c-8e6e-4fca-98f7-1b1b4c7542c5 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:55.901059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:55.904264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62f515-813e-46c1-988b-6257ccb42f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:55.905091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62f515-813e-46c1-988b-6257ccb42f77 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:55.905938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62f515-813e-46c1-988b-6257ccb42f77 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:55.906361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62f515-813e-46c1-988b-6257ccb42f77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:55.922265 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62f515-813e-46c1-988b-6257ccb42f77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:55.924682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62f515-813e-46c1-988b-6257ccb42f77 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-6e62f515-813e-46c1-988b-6257ccb42f77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:55.945391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:55.946583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:55.947645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:55.947645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:55.947645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:55.954381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 323/633] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:55.954381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa4e1c-e740-4bac-acaf-1029bc1db0d9 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:55.954381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa4e1c-e740-4bac-acaf-1029bc1db0d9 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:55.955342 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa4e1c-e740-4bac-acaf-1029bc1db0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:55.974363 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-edfa4e1c-e740-4bac-acaf-1029bc1db0d9 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:55.975422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-edfa4e1c-e740-4bac-acaf-1029bc1db0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:55.982358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df0d3c-8e6e-4fca-98f7-1b1b4c7542c5 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:56.027543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a2878-72fe-4e6b-b5c5-8cd425b69bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:56.028815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a2878-72fe-4e6b-b5c5-8cd425b69bc2 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:56.029529 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a2878-72fe-4e6b-b5c5-8cd425b69bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:56.029980 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a2878-72fe-4e6b-b5c5-8cd425b69bc2 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:56.030946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a2878-72fe-4e6b-b5c5-8cd425b69bc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:56.031683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a2878-72fe-4e6b-b5c5-8cd425b69bc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:56.033263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a2878-72fe-4e6b-b5c5-8cd425b69bc2 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:56.046646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 311/634] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:56.047803 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a2878-72fe-4e6b-b5c5-8cd425b69bc2 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:56.049654 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 324/635] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:56.050433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa4e1c-e740-4bac-acaf-1029bc1db0d9 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:56.133249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:56.162266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a2878-72fe-4e6b-b5c5-8cd425b69bc2 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:56.183992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa4e1c-e740-4bac-acaf-1029bc1db0d9 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:56.189027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:56.204082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc1370-4101-46e7-9a29-8882d018a89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:56.204719 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc1370-4101-46e7-9a29-8882d018a89b None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:56.205008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc1370-4101-46e7-9a29-8882d018a89b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:56.205303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc1370-4101-46e7-9a29-8882d018a89b None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:56.205947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc1370-4101-46e7-9a29-8882d018a89b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:56.206313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc1370-4101-46e7-9a29-8882d018a89b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:56.208299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc1370-4101-46e7-9a29-8882d018a89b None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:56.224613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 312/636] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:56.225489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc1370-4101-46e7-9a29-8882d018a89b None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:56.228701 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324695c-3404-4ab3-9c34-bf733a70737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:56.230710 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324695c-3404-4ab3-9c34-bf733a70737c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:56.231197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324695c-3404-4ab3-9c34-bf733a70737c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:56.231709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324695c-3404-4ab3-9c34-bf733a70737c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:56.245497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324695c-3404-4ab3-9c34-bf733a70737c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-9324695c-3404-4ab3-9c34-bf733a70737c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:56.251567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:56.252470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:56.253346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:56.253346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:56.253346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:56.253346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:56.253346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:56.253346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:56.253346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:56.253346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:56.253346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:56.253346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:56.253346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:56.253346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:56.259850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 325/637] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:56.259850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324695c-3404-4ab3-9c34-bf733a70737c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:56.349483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc1370-4101-46e7-9a29-8882d018a89b None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:56.375138 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324695c-3404-4ab3-9c34-bf733a70737c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:56.384437 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cc8c6-6853-4c4b-bc7f-ac6dfd1d1e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:56.385161 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cc8c6-6853-4c4b-bc7f-ac6dfd1d1e3f None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:56.385443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cc8c6-6853-4c4b-bc7f-ac6dfd1d1e3f None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:56.385713 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cc8c6-6853-4c4b-bc7f-ac6dfd1d1e3f None admin] PATH_INFO: `/v3/users/nova` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:56.393138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cc8c6-6853-4c4b-bc7f-ac6dfd1d1e3f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:56.395302 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cc8c6-6853-4c4b-bc7f-ac6dfd1d1e3f None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:56.400056 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-e31cc8c6-6853-4c4b-bc7f-ac6dfd1d1e3f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:32:56.406770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:56.407704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:56.409180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:56.409180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:56.409180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:32:56.409180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:32:56.409180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:32:56.409180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:32:56.409180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:32:56.409180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:32:56.409180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:32:56.409180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:56.409180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 313/638] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 24 07:32:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:56.409180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cc8c6-6853-4c4b-bc7f-ac6dfd1d1e3f None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:56.416506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f499e-bb87-4d24-98d1-ecbcd059cf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:56.422565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f499e-bb87-4d24-98d1-ecbcd059cf06 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:56.422565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f499e-bb87-4d24-98d1-ecbcd059cf06 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:56.422565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f499e-bb87-4d24-98d1-ecbcd059cf06 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:56.424930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f499e-bb87-4d24-98d1-ecbcd059cf06 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:56.425059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:56.426388 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f499e-bb87-4d24-98d1-ecbcd059cf06 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-e62f499e-bb87-4d24-98d1-ecbcd059cf06 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:32:56.437877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:56.438890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:56.439825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:56.439825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:56.439825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:32:56.439825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:32:56.439825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:32:56.439825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:32:56.439825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:32:56.439825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:32:56.439825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:32:56.439825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:56.439825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 326/639] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:56.439825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f499e-bb87-4d24-98d1-ecbcd059cf06 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:56.520223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cc8c6-6853-4c4b-bc7f-ac6dfd1d1e3f None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:56.548126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f499e-bb87-4d24-98d1-ecbcd059cf06 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:56.558710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c819c-c540-467e-9715-51d908ad43d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:56.559434 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c819c-c540-467e-9715-51d908ad43d9 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:56.559811 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c819c-c540-467e-9715-51d908ad43d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:56.560181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c819c-c540-467e-9715-51d908ad43d9 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:56.560988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c819c-c540-467e-9715-51d908ad43d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:56.561355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c819c-c540-467e-9715-51d908ad43d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:56.562783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c819c-c540-467e-9715-51d908ad43d9 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:56.575418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 314/640] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:56.575418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c819c-c540-467e-9715-51d908ad43d9 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:56.591684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b7830-ff1c-4648-8542-e973b626cf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:56.592325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b7830-ff1c-4648-8542-e973b626cf34 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:56.595445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b7830-ff1c-4648-8542-e973b626cf34 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:56.595445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b7830-ff1c-4648-8542-e973b626cf34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:56.609735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b7830-ff1c-4648-8542-e973b626cf34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-982b7830-ff1c-4648-8542-e973b626cf34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:56.614924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:56.616089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:56.617575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:56.617575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:56.617575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:56.617575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:56.617575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:56.617575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:56.617575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:56.617575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:56.617575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:56.617575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:56.617575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:56.617575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:56.623435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 327/641] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:56.623435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b7830-ff1c-4648-8542-e973b626cf34 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:56.662107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:56.667016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:56.687836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c819c-c540-467e-9715-51d908ad43d9 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:56.730028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b7830-ff1c-4648-8542-e973b626cf34 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:56.734210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aef699-6f79-4e50-8342-1277f40c6612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:56.734887 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aef699-6f79-4e50-8342-1277f40c6612 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:56.735238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aef699-6f79-4e50-8342-1277f40c6612 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:56.735581 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aef699-6f79-4e50-8342-1277f40c6612 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:56.736382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aef699-6f79-4e50-8342-1277f40c6612 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:56.738163 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aef699-6f79-4e50-8342-1277f40c6612 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:56.740932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aef699-6f79-4e50-8342-1277f40c6612 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:56.759277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 315/642] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 24 07:32:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:56.761598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aef699-6f79-4e50-8342-1277f40c6612 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:56.774401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b8dd0-a9fd-4550-90a7-79a71d876629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:56.775164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b8dd0-a9fd-4550-90a7-79a71d876629 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:56.775546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b8dd0-a9fd-4550-90a7-79a71d876629 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:56.775882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b8dd0-a9fd-4550-90a7-79a71d876629 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:56.776743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b8dd0-a9fd-4550-90a7-79a71d876629 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:56.777120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b8dd0-a9fd-4550-90a7-79a71d876629 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:56.778616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b8dd0-a9fd-4550-90a7-79a71d876629 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:56.796362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 328/643] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Aug 24 07:32:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:56.797086 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b8dd0-a9fd-4550-90a7-79a71d876629 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:56.871976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aef699-6f79-4e50-8342-1277f40c6612 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:56.901671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:56.903138 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:56.906332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b8dd0-a9fd-4550-90a7-79a71d876629 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:56.917884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea9786-5a02-484f-bd5a-dc9e1189f70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:56.917884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea9786-5a02-484f-bd5a-dc9e1189f70d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:56.917884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea9786-5a02-484f-bd5a-dc9e1189f70d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:56.917884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea9786-5a02-484f-bd5a-dc9e1189f70d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:56.918248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea9786-5a02-484f-bd5a-dc9e1189f70d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:56.919320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea9786-5a02-484f-bd5a-dc9e1189f70d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-c3ea9786-5a02-484f-bd5a-dc9e1189f70d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:56.934791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:56.935764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:56.937445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:56.937445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:56.937445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:56.937445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 316/644] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:56.937445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea9786-5a02-484f-bd5a-dc9e1189f70d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:56.947394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7fcb5-db29-4513-a2dc-969eb32fecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:56.951705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7fcb5-db29-4513-a2dc-969eb32fecc6 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:56.951705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7fcb5-db29-4513-a2dc-969eb32fecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:56.951705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7fcb5-db29-4513-a2dc-969eb32fecc6 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:56.951705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7fcb5-db29-4513-a2dc-969eb32fecc6 None admin] RBAC: Adding query filter params (name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:56.951705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7fcb5-db29-4513-a2dc-969eb32fecc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:56.951705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7fcb5-db29-4513-a2dc-969eb32fecc6 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:56.965275 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 329/645] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:56.965275 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7fcb5-db29-4513-a2dc-969eb32fecc6 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:57.046511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea9786-5a02-484f-bd5a-dc9e1189f70d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:57.073829 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7fcb5-db29-4513-a2dc-969eb32fecc6 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:57.085021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a95f96-c587-46fb-a55e-2c4e29f9b4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:57.085633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a95f96-c587-46fb-a55e-2c4e29f9b4a0 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:57.085913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a95f96-c587-46fb-a55e-2c4e29f9b4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:57.086222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a95f96-c587-46fb-a55e-2c4e29f9b4a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:57.110550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf4b7a-3ef3-47c0-a2a2-dc1e8b6b08f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:57.111157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf4b7a-3ef3-47c0-a2a2-dc1e8b6b08f2 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:57.111439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf4b7a-3ef3-47c0-a2a2-dc1e8b6b08f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:57.111717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf4b7a-3ef3-47c0-a2a2-dc1e8b6b08f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:57.142033 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:57.143795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:57.158486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a95f96-c587-46fb-a55e-2c4e29f9b4a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:57.160619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a95f96-c587-46fb-a55e-2c4e29f9b4a0 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:57.183171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf4b7a-3ef3-47c0-a2a2-dc1e8b6b08f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:57.184625 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf4b7a-3ef3-47c0-a2a2-dc1e8b6b08f2 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:57.267782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 317/646] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:57.268935 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a95f96-c587-46fb-a55e-2c4e29f9b4a0 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:57.296682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 330/647] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:57.297202 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf4b7a-3ef3-47c0-a2a2-dc1e8b6b08f2 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:57.361260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:57.364006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:57.376150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a95f96-c587-46fb-a55e-2c4e29f9b4a0 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:57.403180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf4b7a-3ef3-47c0-a2a2-dc1e8b6b08f2 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:57.414021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6019ac-9ff6-4417-804c-81a3d237abbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:57.414021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6019ac-9ff6-4417-804c-81a3d237abbf None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:57.414254 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6019ac-9ff6-4417-804c-81a3d237abbf None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:57.414400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6019ac-9ff6-4417-804c-81a3d237abbf None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:57.415037 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6019ac-9ff6-4417-804c-81a3d237abbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:57.415339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6019ac-9ff6-4417-804c-81a3d237abbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:57.416723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6019ac-9ff6-4417-804c-81a3d237abbf None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:57.428369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 318/648] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:57.429375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6019ac-9ff6-4417-804c-81a3d237abbf None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:57.445049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71a6cd-a4fb-4278-89a7-3e664fbd1462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:57.445674 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71a6cd-a4fb-4278-89a7-3e664fbd1462 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:57.445953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71a6cd-a4fb-4278-89a7-3e664fbd1462 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:57.446226 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71a6cd-a4fb-4278-89a7-3e664fbd1462 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:57.458345 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71a6cd-a4fb-4278-89a7-3e664fbd1462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:57.459827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71a6cd-a4fb-4278-89a7-3e664fbd1462 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-ae71a6cd-a4fb-4278-89a7-3e664fbd1462 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:57.474747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:57.475883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:57.481951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:57.481951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:57.481951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:57.481951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 331/649] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:57.481951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71a6cd-a4fb-4278-89a7-3e664fbd1462 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:57.540190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6019ac-9ff6-4417-804c-81a3d237abbf None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:57.578113 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6a081-78d0-4263-8907-0ff9d0d15009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:57.578752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6a081-78d0-4263-8907-0ff9d0d15009 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:57.579011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6a081-78d0-4263-8907-0ff9d0d15009 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:57.579285 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6a081-78d0-4263-8907-0ff9d0d15009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:57.588509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71a6cd-a4fb-4278-89a7-3e664fbd1462 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:57.590758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6a081-78d0-4263-8907-0ff9d0d15009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:57.592197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6a081-78d0-4263-8907-0ff9d0d15009 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:57.595075 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:57.602740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-2cd6a081-78d0-4263-8907-0ff9d0d15009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:57.607667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:57.608754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:57.609632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:57.609632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:57.609632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:57.609632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 319/650] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:57.609632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6a081-78d0-4263-8907-0ff9d0d15009 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:57.626885 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac99ee1-3f40-4457-9a05-51a1fc22986c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:57.627511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac99ee1-3f40-4457-9a05-51a1fc22986c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:57.627793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac99ee1-3f40-4457-9a05-51a1fc22986c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:57.628063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac99ee1-3f40-4457-9a05-51a1fc22986c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:57.639043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac99ee1-3f40-4457-9a05-51a1fc22986c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:57.640465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac99ee1-3f40-4457-9a05-51a1fc22986c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-5ac99ee1-3f40-4457-9a05-51a1fc22986c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:57.654215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:57.655298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:57.656393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:57.656393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:57.656393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:57.660968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 332/651] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:57.661873 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac99ee1-3f40-4457-9a05-51a1fc22986c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:57.711620 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6a081-78d0-4263-8907-0ff9d0d15009 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:57.745479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a33d4-1268-49f9-8c49-67afe9fa1e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:57.746102 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a33d4-1268-49f9-8c49-67afe9fa1e97 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:57.746381 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a33d4-1268-49f9-8c49-67afe9fa1e97 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:57.746656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a33d4-1268-49f9-8c49-67afe9fa1e97 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:57.754244 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a33d4-1268-49f9-8c49-67afe9fa1e97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:57.755681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a33d4-1268-49f9-8c49-67afe9fa1e97 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:57.760355 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac99ee1-3f40-4457-9a05-51a1fc22986c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-ab1a33d4-1268-49f9-8c49-67afe9fa1e97 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:32:57.766693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:57.767833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:57.768878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:57.768878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:57.768878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:32:57.768878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:32:57.768878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:32:57.768878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:32:57.768878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:32:57.768878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:32:57.768878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:32:57.768878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:57.768878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 320/652] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:32:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:57.768878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d98313-9360-4711-a976-f580faf84cf9 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:57.769752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d98313-9360-4711-a976-f580faf84cf9 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:57.769752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d98313-9360-4711-a976-f580faf84cf9 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:57.774709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 321/653] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:32:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:32:57.774709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d98313-9360-4711-a976-f580faf84cf9 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:57.797136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea71ec9-5ed8-433a-b430-3e54d0753336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:57.797740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea71ec9-5ed8-433a-b430-3e54d0753336 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:57.798021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea71ec9-5ed8-433a-b430-3e54d0753336 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:57.798301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea71ec9-5ed8-433a-b430-3e54d0753336 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:57.798935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea71ec9-5ed8-433a-b430-3e54d0753336 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:57.799240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea71ec9-5ed8-433a-b430-3e54d0753336 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:57.800597 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea71ec9-5ed8-433a-b430-3e54d0753336 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:57.813103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 333/654] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:57.813875 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea71ec9-5ed8-433a-b430-3e54d0753336 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:57.832547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:57.843240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:57.875646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d98313-9360-4711-a976-f580faf84cf9 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:57.909097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8358be-4b50-474e-a377-ad72824739e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:57.909623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8358be-4b50-474e-a377-ad72824739e9 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:57.909898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8358be-4b50-474e-a377-ad72824739e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:57.910683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8358be-4b50-474e-a377-ad72824739e9 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:57.911313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8358be-4b50-474e-a377-ad72824739e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:57.911612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8358be-4b50-474e-a377-ad72824739e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:57.912990 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8358be-4b50-474e-a377-ad72824739e9 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:57.914629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea71ec9-5ed8-433a-b430-3e54d0753336 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:57.924351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 322/655] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:57.925165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8358be-4b50-474e-a377-ad72824739e9 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:57.950602 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd954c-5d05-449d-a0f6-ccc6a55f2089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:57.951211 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd954c-5d05-449d-a0f6-ccc6a55f2089 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:57.951493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd954c-5d05-449d-a0f6-ccc6a55f2089 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:57.951767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd954c-5d05-449d-a0f6-ccc6a55f2089 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:57.952407 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd954c-5d05-449d-a0f6-ccc6a55f2089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:57.952751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd954c-5d05-449d-a0f6-ccc6a55f2089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:57.954133 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd954c-5d05-449d-a0f6-ccc6a55f2089 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:57.965553 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 334/656] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:57.966706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efdec7-75c2-4e19-a57c-6c5e1bcd192f None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:57.966996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efdec7-75c2-4e19-a57c-6c5e1bcd192f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:57.967276 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efdec7-75c2-4e19-a57c-6c5e1bcd192f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:57.985117 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a6efdec7-75c2-4e19-a57c-6c5e1bcd192f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:32:57.990518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a6efdec7-75c2-4e19-a57c-6c5e1bcd192f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:32:58.027648 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8358be-4b50-474e-a377-ad72824739e9 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:58.045564 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:58.053169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 335/657] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:32:58.054118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efdec7-75c2-4e19-a57c-6c5e1bcd192f None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:58.066862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dc0d9-5df3-4ebe-bc1d-9d4767eca975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:58.067526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dc0d9-5df3-4ebe-bc1d-9d4767eca975 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:58.067812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dc0d9-5df3-4ebe-bc1d-9d4767eca975 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:58.068085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dc0d9-5df3-4ebe-bc1d-9d4767eca975 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:58.068912 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545dc0d9-5df3-4ebe-bc1d-9d4767eca975 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:58.069224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545dc0d9-5df3-4ebe-bc1d-9d4767eca975 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:58.070642 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545dc0d9-5df3-4ebe-bc1d-9d4767eca975 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:58.071256 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:58.088572 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 323/658] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Aug 24 07:32:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:58.089475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dc0d9-5df3-4ebe-bc1d-9d4767eca975 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:58.165344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efdec7-75c2-4e19-a57c-6c5e1bcd192f None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:58.194818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dc0d9-5df3-4ebe-bc1d-9d4767eca975 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:58.202252 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0eb9dc-2f62-4fb7-b4af-ca9b006663fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:58.202987 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0eb9dc-2f62-4fb7-b4af-ca9b006663fb None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:58.203365 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0eb9dc-2f62-4fb7-b4af-ca9b006663fb None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:58.203752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0eb9dc-2f62-4fb7-b4af-ca9b006663fb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:58.215953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0eb9dc-2f62-4fb7-b4af-ca9b006663fb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:58.217881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0eb9dc-2f62-4fb7-b4af-ca9b006663fb None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-7b0eb9dc-2f62-4fb7-b4af-ca9b006663fb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:32:58.229969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:58.231101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:58.232230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:58.232230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:58.232230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:32:58.232230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:32:58.232230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:32:58.232230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:32:58.232230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:32:58.232230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:32:58.232230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:32:58.232230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:58.233556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b93d31-b153-4611-960a-25fc73b11c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:58.234280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93d31-b153-4611-960a-25fc73b11c94 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:58.234653 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93d31-b153-4611-960a-25fc73b11c94 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:58.235019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93d31-b153-4611-960a-25fc73b11c94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:58.237576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 336/659] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:58.238835 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0eb9dc-2f62-4fb7-b4af-ca9b006663fb None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:58.243415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b93d31-b153-4611-960a-25fc73b11c94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:58.244993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b93d31-b153-4611-960a-25fc73b11c94 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-b1b93d31-b153-4611-960a-25fc73b11c94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:32:58.256347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:58.257442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:58.258370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:32:58.258370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:32:58.258370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:32:58.258370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:32:58.258370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:32:58.258370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:32:58.258370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:58.258370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:58.258370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 324/660] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:58.258848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b93d31-b153-4611-960a-25fc73b11c94 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:58.293877 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:58.331538 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:58.359321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0eb9dc-2f62-4fb7-b4af-ca9b006663fb None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:58.370076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b93d31-b153-4611-960a-25fc73b11c94 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:58.394780 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16928f8e-64da-4d44-a90c-ac1dcd3eaa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:58.395440 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16928f8e-64da-4d44-a90c-ac1dcd3eaa4f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:58.395723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16928f8e-64da-4d44-a90c-ac1dcd3eaa4f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:58.396000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16928f8e-64da-4d44-a90c-ac1dcd3eaa4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:58.402664 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16928f8e-64da-4d44-a90c-ac1dcd3eaa4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:58.404093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16928f8e-64da-4d44-a90c-ac1dcd3eaa4f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:58.404521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6a0c0-fa26-4c1d-a807-c9e0c58265e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:58.405146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6a0c0-fa26-4c1d-a807-c9e0c58265e5 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:58.405426 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6a0c0-fa26-4c1d-a807-c9e0c58265e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:58.405697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6a0c0-fa26-4c1d-a807-c9e0c58265e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-16928f8e-64da-4d44-a90c-ac1dcd3eaa4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:32:58.415415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:58.416524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:58.417532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:32:58.417532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:32:58.417532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:32:58.417532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:32:58.417532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:32:58.417532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:32:58.417532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:58.417532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:58.417532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 337/661] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:58.417532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16928f8e-64da-4d44-a90c-ac1dcd3eaa4f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:58.476199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6a0c0-fa26-4c1d-a807-c9e0c58265e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:58.477688 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6a0c0-fa26-4c1d-a807-c9e0c58265e5 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:58.509617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:58.516588 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16928f8e-64da-4d44-a90c-ac1dcd3eaa4f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:58.549768 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055132a-21ee-411e-b2e1-86ad4bd09fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:58.550372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055132a-21ee-411e-b2e1-86ad4bd09fad None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:58.550656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055132a-21ee-411e-b2e1-86ad4bd09fad None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:58.551570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055132a-21ee-411e-b2e1-86ad4bd09fad None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:58.552301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055132a-21ee-411e-b2e1-86ad4bd09fad None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:58.552604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055132a-21ee-411e-b2e1-86ad4bd09fad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:58.554059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055132a-21ee-411e-b2e1-86ad4bd09fad None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:58.556568 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:58.571020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 338/662] 10.222.0.26 () {56 vars in 1139 bytes} [Wed Aug 24 07:32:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:58.572027 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055132a-21ee-411e-b2e1-86ad4bd09fad None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:58.575974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 325/663] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:58.579781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6a0c0-fa26-4c1d-a807-c9e0c58265e5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:58.670233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055132a-21ee-411e-b2e1-86ad4bd09fad None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:58.674847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6a0c0-fa26-4c1d-a807-c9e0c58265e5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:58.702716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd921d-08ec-48ac-a8bc-d60654c8475e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:58.703312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd921d-08ec-48ac-a8bc-d60654c8475e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:58.703603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd921d-08ec-48ac-a8bc-d60654c8475e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:58.703873 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd921d-08ec-48ac-a8bc-d60654c8475e None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:58.704555 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd921d-08ec-48ac-a8bc-d60654c8475e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:58.704889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd921d-08ec-48ac-a8bc-d60654c8475e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:58.706423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd921d-08ec-48ac-a8bc-d60654c8475e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:58.708320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e5a63-8b8d-497a-8e5c-7cd227d4335d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:58.708969 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e5a63-8b8d-497a-8e5c-7cd227d4335d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:58.709248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e5a63-8b8d-497a-8e5c-7cd227d4335d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:58.709521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e5a63-8b8d-497a-8e5c-7cd227d4335d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:58.711272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:58.717525 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 339/664] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:32:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:58.718391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd921d-08ec-48ac-a8bc-d60654c8475e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:58.723447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e5a63-8b8d-497a-8e5c-7cd227d4335d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-d61e5a63-8b8d-497a-8e5c-7cd227d4335d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:32:58.725489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:58.726625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:58.727551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:58.727551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:58.727551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:58.727551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:58.727551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:32:58.727551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:32:58.727551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:32:58.727551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:32:58.727551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:32:58.727551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:32:58.727551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:32:58.727551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:32:58.727551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 326/665] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:32:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:58.727551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e5a63-8b8d-497a-8e5c-7cd227d4335d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:58.782245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:58.826772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e5a63-8b8d-497a-8e5c-7cd227d4335d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:58.831291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd921d-08ec-48ac-a8bc-d60654c8475e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:58.860665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c2eb7-94c3-4336-8ed0-a9340a04a12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:58.861293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c2eb7-94c3-4336-8ed0-a9340a04a12a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:58.861571 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c2eb7-94c3-4336-8ed0-a9340a04a12a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:58.861847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c2eb7-94c3-4336-8ed0-a9340a04a12a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:58.869566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df5ce3-0e46-4a55-bed0-1a74e23a846b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:58.870203 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df5ce3-0e46-4a55-bed0-1a74e23a846b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:58.870481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df5ce3-0e46-4a55-bed0-1a74e23a846b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:58.870748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df5ce3-0e46-4a55-bed0-1a74e23a846b None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:58.871428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df5ce3-0e46-4a55-bed0-1a74e23a846b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:58.871730 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df5ce3-0e46-4a55-bed0-1a74e23a846b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:58.873188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df5ce3-0e46-4a55-bed0-1a74e23a846b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:58.885320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 340/666] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:32:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:58.886214 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df5ce3-0e46-4a55-bed0-1a74e23a846b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:58.920701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:58.927782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c2eb7-94c3-4336-8ed0-a9340a04a12a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:58.929301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c2eb7-94c3-4336-8ed0-a9340a04a12a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:58.986482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df5ce3-0e46-4a55-bed0-1a74e23a846b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:59.000096 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:59.020899 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442f88f-0ebe-4331-9364-05bf04575178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:59.021522 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442f88f-0ebe-4331-9364-05bf04575178 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:59.021800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442f88f-0ebe-4331-9364-05bf04575178 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:59.022072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442f88f-0ebe-4331-9364-05bf04575178 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:59.026453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 327/667] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:32:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:32:59.027286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c2eb7-94c3-4336-8ed0-a9340a04a12a None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:59.034738 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442f88f-0ebe-4331-9364-05bf04575178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:59.037085 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442f88f-0ebe-4331-9364-05bf04575178 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-9442f88f-0ebe-4331-9364-05bf04575178 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:59.051542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:59.052562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:59.053483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:59.053483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:59.053483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:59.053483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 341/668] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:59.053483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442f88f-0ebe-4331-9364-05bf04575178 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:59.121467 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:59.127830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c2eb7-94c3-4336-8ed0-a9340a04a12a None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:59.157459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442f88f-0ebe-4331-9364-05bf04575178 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:59.162512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180f96c-e0e6-4dc3-943b-ca3b0902358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:59.163179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180f96c-e0e6-4dc3-943b-ca3b0902358b None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:59.163529 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180f96c-e0e6-4dc3-943b-ca3b0902358b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:59.163867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180f96c-e0e6-4dc3-943b-ca3b0902358b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:59.164673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180f96c-e0e6-4dc3-943b-ca3b0902358b None admin] RBAC: Adding query filter params (role.id=3acdf08298c84325b53a0e286cc53b30, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=bdf81f09006a42299c0807f77f16c748) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:59.164996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180f96c-e0e6-4dc3-943b-ca3b0902358b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:59.166440 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180f96c-e0e6-4dc3-943b-ca3b0902358b None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:59.172438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 328/669] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:32:59 2022] GET /identity/v3/role_assignments?user.id=bdf81f09006a42299c0807f77f16c748&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=3acdf08298c84325b53a0e286cc53b30 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:59.173201 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180f96c-e0e6-4dc3-943b-ca3b0902358b None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:59.213650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fca25-d925-4e3f-9d80-ffcdcfd15a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:59.214271 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fca25-d925-4e3f-9d80-ffcdcfd15a14 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:59.214560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fca25-d925-4e3f-9d80-ffcdcfd15a14 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:59.214830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fca25-d925-4e3f-9d80-ffcdcfd15a14 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:59.215495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fca25-d925-4e3f-9d80-ffcdcfd15a14 None admin] RBAC: Adding query filter params (name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:59.215798 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fca25-d925-4e3f-9d80-ffcdcfd15a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:59.217267 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fca25-d925-4e3f-9d80-ffcdcfd15a14 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:59.230993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 342/670] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:32:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:59.230993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fca25-d925-4e3f-9d80-ffcdcfd15a14 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:59.261496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:59.293928 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180f96c-e0e6-4dc3-943b-ca3b0902358b None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:59.339125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42533ea8-e8f9-4154-96c0-a0cb7dd3d17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:59.339807 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42533ea8-e8f9-4154-96c0-a0cb7dd3d17f None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:59.340748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42533ea8-e8f9-4154-96c0-a0cb7dd3d17f None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:59.340748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42533ea8-e8f9-4154-96c0-a0cb7dd3d17f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:59.341590 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42533ea8-e8f9-4154-96c0-a0cb7dd3d17f None admin] RBAC: Adding query filter params (role.id=3acdf08298c84325b53a0e286cc53b30, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=43e57131a5be4af785166bc03a3337a2) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:59.341590 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42533ea8-e8f9-4154-96c0-a0cb7dd3d17f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:59.342352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fca25-d925-4e3f-9d80-ffcdcfd15a14 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:59.342846 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42533ea8-e8f9-4154-96c0-a0cb7dd3d17f None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:59.350501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 329/671] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:32:59 2022] GET /identity/v3/role_assignments?user.id=43e57131a5be4af785166bc03a3337a2&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=3acdf08298c84325b53a0e286cc53b30 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:59.353736 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42533ea8-e8f9-4154-96c0-a0cb7dd3d17f None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:59.382479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:59.387701 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601ef45-bc8f-4878-85e2-b8bfe274fdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:59.389123 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601ef45-bc8f-4878-85e2-b8bfe274fdb7 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:59.389389 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601ef45-bc8f-4878-85e2-b8bfe274fdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:59.389671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601ef45-bc8f-4878-85e2-b8bfe274fdb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:59.402366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601ef45-bc8f-4878-85e2-b8bfe274fdb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:59.403989 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601ef45-bc8f-4878-85e2-b8bfe274fdb7 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-5601ef45-bc8f-4878-85e2-b8bfe274fdb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:59.425260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:59.426317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:59.427500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:59.427500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:59.427500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:59.427500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 343/672] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:59.427500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601ef45-bc8f-4878-85e2-b8bfe274fdb7 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:59.498169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42533ea8-e8f9-4154-96c0-a0cb7dd3d17f None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:59.530876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:59.548521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e242d-b723-4c46-88da-55b67f5dce57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:59.549162 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e242d-b723-4c46-88da-55b67f5dce57 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:59.549450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e242d-b723-4c46-88da-55b67f5dce57 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:59.549725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e242d-b723-4c46-88da-55b67f5dce57 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:59.550374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e242d-b723-4c46-88da-55b67f5dce57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:59.551299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e242d-b723-4c46-88da-55b67f5dce57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:59.552880 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e242d-b723-4c46-88da-55b67f5dce57 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:59.556753 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601ef45-bc8f-4878-85e2-b8bfe274fdb7 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:59.569595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 330/673] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:59.570542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e242d-b723-4c46-88da-55b67f5dce57 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:59.603150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d720f-95fe-49ad-8e15-b00f6cc7bc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:59.604191 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d720f-95fe-49ad-8e15-b00f6cc7bc2c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:59.604470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d720f-95fe-49ad-8e15-b00f6cc7bc2c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:59.604769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d720f-95fe-49ad-8e15-b00f6cc7bc2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:59.619352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d720f-95fe-49ad-8e15-b00f6cc7bc2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:59.621650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d720f-95fe-49ad-8e15-b00f6cc7bc2c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-eb6d720f-95fe-49ad-8e15-b00f6cc7bc2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:32:59.644321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:59.645901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:32:59.646825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:32:59.646825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:32:59.646825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:59.648988 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 344/674] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:59.649926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d720f-95fe-49ad-8e15-b00f6cc7bc2c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:59.672511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:59.694478 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e242d-b723-4c46-88da-55b67f5dce57 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:59.731589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c885c-b694-4b80-94a7-13b736ca876c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:59.732348 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c885c-b694-4b80-94a7-13b736ca876c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:59.732752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c885c-b694-4b80-94a7-13b736ca876c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:59.733122 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c885c-b694-4b80-94a7-13b736ca876c None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:59.733887 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c885c-b694-4b80-94a7-13b736ca876c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:59.734289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c885c-b694-4b80-94a7-13b736ca876c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:59.735749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c885c-b694-4b80-94a7-13b736ca876c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:59.748065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 331/675] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:59.753188 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c885c-b694-4b80-94a7-13b736ca876c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:59.769060 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d720f-95fe-49ad-8e15-b00f6cc7bc2c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:59.802692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:59.808210 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e8687-2e1d-4041-92d3-521bcec0bb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:59.809138 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e8687-2e1d-4041-92d3-521bcec0bb81 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:59.809567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e8687-2e1d-4041-92d3-521bcec0bb81 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:59.810167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e8687-2e1d-4041-92d3-521bcec0bb81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:59.817351 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e8687-2e1d-4041-92d3-521bcec0bb81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:59.821396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e8687-2e1d-4041-92d3-521bcec0bb81 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-cb5e8687-2e1d-4041-92d3-521bcec0bb81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:32:59.831161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:32:59.832626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:32:59.833922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:32:59.833922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:32:59.833922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:32:59.833922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:32:59.833922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:32:59.833922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:32:59.833922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:32:59.833922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:32:59.836721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 345/676] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:32:59.837761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e8687-2e1d-4041-92d3-521bcec0bb81 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:59.876283 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c885c-b694-4b80-94a7-13b736ca876c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:59.918430 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67813aea-85dc-4fa3-b9a1-0558a09f8c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:59.919041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67813aea-85dc-4fa3-b9a1-0558a09f8c4d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:59.919322 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67813aea-85dc-4fa3-b9a1-0558a09f8c4d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:59.919598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67813aea-85dc-4fa3-b9a1-0558a09f8c4d None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:59.920236 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67813aea-85dc-4fa3-b9a1-0558a09f8c4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:32:59.920542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67813aea-85dc-4fa3-b9a1-0558a09f8c4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:59.921957 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67813aea-85dc-4fa3-b9a1-0558a09f8c4d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:32:59.935151 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 332/677] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:32:59.936049 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67813aea-85dc-4fa3-b9a1-0558a09f8c4d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:32:59.938879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:32:59.952895 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e8687-2e1d-4041-92d3-521bcec0bb81 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:32:59.988578 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfae83-e512-4bcd-8e40-f76764a41320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:32:59.989498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfae83-e512-4bcd-8e40-f76764a41320 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:32:59.989788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfae83-e512-4bcd-8e40-f76764a41320 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:32:59.990113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfae83-e512-4bcd-8e40-f76764a41320 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:32:59.997549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfae83-e512-4bcd-8e40-f76764a41320 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:32:59.999113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfae83-e512-4bcd-8e40-f76764a41320 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-f6dfae83-e512-4bcd-8e40-f76764a41320 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:00.012079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:00.016102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:00.017235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:00.017235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:00.017235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:00.017235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:00.017235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:00.017235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:00.017235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:00.017235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:00.017235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 346/678] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:32:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:00.017235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfae83-e512-4bcd-8e40-f76764a41320 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:00.047380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67813aea-85dc-4fa3-b9a1-0558a09f8c4d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:00.073863 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:00.085393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bb9b5-858f-4ce9-ac65-6f0b1d247c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:00.090230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bb9b5-858f-4ce9-ac65-6f0b1d247c1a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:00.090230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bb9b5-858f-4ce9-ac65-6f0b1d247c1a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:00.090230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bb9b5-858f-4ce9-ac65-6f0b1d247c1a None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:00.090230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46bb9b5-858f-4ce9-ac65-6f0b1d247c1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:00.090230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46bb9b5-858f-4ce9-ac65-6f0b1d247c1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:00.090230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46bb9b5-858f-4ce9-ac65-6f0b1d247c1a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:00.102568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 333/679] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:32:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:00.103653 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bb9b5-858f-4ce9-ac65-6f0b1d247c1a None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:00.131582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfae83-e512-4bcd-8e40-f76764a41320 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:00.168490 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698be3c-e2a2-41bb-bbe8-1c767ae12c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:00.169300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698be3c-e2a2-41bb-bbe8-1c767ae12c1b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:00.169933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:00.170055 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698be3c-e2a2-41bb-bbe8-1c767ae12c1b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:00.171330 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698be3c-e2a2-41bb-bbe8-1c767ae12c1b None admin] PATH_INFO: `/v3/users/swift` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:00.179159 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698be3c-e2a2-41bb-bbe8-1c767ae12c1b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:00.181955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698be3c-e2a2-41bb-bbe8-1c767ae12c1b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-f698be3c-e2a2-41bb-bbe8-1c767ae12c1b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:00.192797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:00.193861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:00.194810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:00.194810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:00.194810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:00.194810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:00.194810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:00.194810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:00.194810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:00.194810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:00.194810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:33:00.194810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:00.194810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 347/680] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:33:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:00.195505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698be3c-e2a2-41bb-bbe8-1c767ae12c1b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:00.216291 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bb9b5-858f-4ce9-ac65-6f0b1d247c1a None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:00.252682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08885c88-1b14-43f1-a065-03101fc635af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:00.253547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08885c88-1b14-43f1-a065-03101fc635af None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:00.254513 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08885c88-1b14-43f1-a065-03101fc635af None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:00.254973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08885c88-1b14-43f1-a065-03101fc635af None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:00.255814 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08885c88-1b14-43f1-a065-03101fc635af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:00.256438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08885c88-1b14-43f1-a065-03101fc635af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:00.258130 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08885c88-1b14-43f1-a065-03101fc635af None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:00.272248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 334/681] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:00.273347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08885c88-1b14-43f1-a065-03101fc635af None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:00.308451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698be3c-e2a2-41bb-bbe8-1c767ae12c1b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:00.323055 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:00.345412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae22571-2bf3-430b-b887-4f109b530415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:00.346101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae22571-2bf3-430b-b887-4f109b530415 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:00.346335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae22571-2bf3-430b-b887-4f109b530415 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:00.346573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae22571-2bf3-430b-b887-4f109b530415 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:00.347307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae22571-2bf3-430b-b887-4f109b530415 None admin] RBAC: Adding query filter params (role.id=3acdf08298c84325b53a0e286cc53b30, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=38de805ba5eb4aa58cf95d2082c2d216) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:00.347891 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae22571-2bf3-430b-b887-4f109b530415 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:00.349450 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae22571-2bf3-430b-b887-4f109b530415 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:00.362874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 348/682] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:00 2022] GET /identity/v3/role_assignments?user.id=38de805ba5eb4aa58cf95d2082c2d216&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=3acdf08298c84325b53a0e286cc53b30 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:00.364977 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae22571-2bf3-430b-b887-4f109b530415 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:00.387677 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08885c88-1b14-43f1-a065-03101fc635af None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:00.415526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:00.429040 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94b5f0-ac1f-4903-a4b7-91b2493a0e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:00.430043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94b5f0-ac1f-4903-a4b7-91b2493a0e5a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:00.430489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94b5f0-ac1f-4903-a4b7-91b2493a0e5a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:00.432335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94b5f0-ac1f-4903-a4b7-91b2493a0e5a None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:00.432335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94b5f0-ac1f-4903-a4b7-91b2493a0e5a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:00.432335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94b5f0-ac1f-4903-a4b7-91b2493a0e5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:00.434139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94b5f0-ac1f-4903-a4b7-91b2493a0e5a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:00.454697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 335/683] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Aug 24 07:33:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:00.461421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94b5f0-ac1f-4903-a4b7-91b2493a0e5a None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:00.586305 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94b5f0-ac1f-4903-a4b7-91b2493a0e5a None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:00.615610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae22571-2bf3-430b-b887-4f109b530415 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:00.626986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd6b61-1a61-4757-b196-6b845e7fc1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:00.627586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd6b61-1a61-4757-b196-6b845e7fc1e0 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:00.627871 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd6b61-1a61-4757-b196-6b845e7fc1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:00.628146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd6b61-1a61-4757-b196-6b845e7fc1e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:00.665822 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d8eac-aa6a-43c8-b172-719ef5a8b213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:00.667124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d8eac-aa6a-43c8-b172-719ef5a8b213 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:00.667412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d8eac-aa6a-43c8-b172-719ef5a8b213 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:00.667692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d8eac-aa6a-43c8-b172-719ef5a8b213 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:00.668460 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d8eac-aa6a-43c8-b172-719ef5a8b213 None admin] RBAC: Adding query filter params (role.id=3acdf08298c84325b53a0e286cc53b30, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=46e61402c01c4f9ba5f18e94960bc313) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:00.668818 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d8eac-aa6a-43c8-b172-719ef5a8b213 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:00.670243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d8eac-aa6a-43c8-b172-719ef5a8b213 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:00.679432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 349/684] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:00 2022] GET /identity/v3/role_assignments?user.id=46e61402c01c4f9ba5f18e94960bc313&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=3acdf08298c84325b53a0e286cc53b30 => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:00.708272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:00.719311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd6b61-1a61-4757-b196-6b845e7fc1e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:00.737090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd6b61-1a61-4757-b196-6b845e7fc1e0 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:00.838143 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 336/685] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:33:00.843977 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d8eac-aa6a-43c8-b172-719ef5a8b213 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:00.897613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:00.959470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d8eac-aa6a-43c8-b172-719ef5a8b213 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:01.004083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03e034-9e4b-4014-bbd5-e5a6d755db47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:01.005181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03e034-9e4b-4014-bbd5-e5a6d755db47 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:01.005693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03e034-9e4b-4014-bbd5-e5a6d755db47 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:01.006244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03e034-9e4b-4014-bbd5-e5a6d755db47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:01.018585 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03e034-9e4b-4014-bbd5-e5a6d755db47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:01.020372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03e034-9e4b-4014-bbd5-e5a6d755db47 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-fe03e034-9e4b-4014-bbd5-e5a6d755db47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:01.037052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:01.038088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:01.039007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:01.039007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:01.039007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:01.040947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 350/686] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:01.047496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd6b61-1a61-4757-b196-6b845e7fc1e0 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:01.157075 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:01.177417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd6b61-1a61-4757-b196-6b845e7fc1e0 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:01.231452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65cf85-541f-4d9a-b063-e0838aeca78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:01.232070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65cf85-541f-4d9a-b063-e0838aeca78d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:01.232350 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65cf85-541f-4d9a-b063-e0838aeca78d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:01.232623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65cf85-541f-4d9a-b063-e0838aeca78d None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:01.233290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65cf85-541f-4d9a-b063-e0838aeca78d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:01.233594 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65cf85-541f-4d9a-b063-e0838aeca78d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:01.234976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65cf85-541f-4d9a-b063-e0838aeca78d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:01.249584 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 337/687] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:01.255599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03e034-9e4b-4014-bbd5-e5a6d755db47 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:01.357231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:01.364558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03e034-9e4b-4014-bbd5-e5a6d755db47 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:01.399712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92174f9-a09a-4b78-add6-a940bac958d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:01.400350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92174f9-a09a-4b78-add6-a940bac958d8 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:01.400627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92174f9-a09a-4b78-add6-a940bac958d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:01.400931 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92174f9-a09a-4b78-add6-a940bac958d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:01.407924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92174f9-a09a-4b78-add6-a940bac958d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:01.409331 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92174f9-a09a-4b78-add6-a940bac958d8 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-c92174f9-a09a-4b78-add6-a940bac958d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:01.420883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:01.421830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:01.422734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:01.422734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:01.422734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:01.422734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:01.422734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:01.422734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:01.422734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:01.422734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:01.422734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 351/688] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:01.424716 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23541f-9de8-47df-b522-020f92a0aaff None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:01.425214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23541f-9de8-47df-b522-020f92a0aaff None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:01.426209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23541f-9de8-47df-b522-020f92a0aaff None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:01.428083 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 338/689] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:01.428195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92174f9-a09a-4b78-add6-a940bac958d8 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:01.433230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca885ff-d41b-42f5-9944-c1a18341facf None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:01.433797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca885ff-d41b-42f5-9944-c1a18341facf None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:01.434272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca885ff-d41b-42f5-9944-c1a18341facf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:01.464905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-9ca885ff-d41b-42f5-9944-c1a18341facf None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:01.466430 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9ca885ff-d41b-42f5-9944-c1a18341facf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:01.538898 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92174f9-a09a-4b78-add6-a940bac958d8 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:01.575336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddfdbf-c145-46a3-8833-9f000866c742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:01.576038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddfdbf-c145-46a3-8833-9f000866c742 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:01.576317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddfdbf-c145-46a3-8833-9f000866c742 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:01.576594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddfdbf-c145-46a3-8833-9f000866c742 None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:01.577317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddfdbf-c145-46a3-8833-9f000866c742 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:01.577618 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddfdbf-c145-46a3-8833-9f000866c742 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:01.579262 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddfdbf-c145-46a3-8833-9f000866c742 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:01.591841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 352/690] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:01.592973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 339/691] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:33:01.598868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e60fc-8810-42d7-82bc-df42567945aa None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:01.599265 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e60fc-8810-42d7-82bc-df42567945aa None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:01.599504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca885ff-d41b-42f5-9944-c1a18341facf None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:01.599610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e60fc-8810-42d7-82bc-df42567945aa None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:01.601154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 353/692] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:01.614718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1f0d4-cc44-4281-bd5b-26aa6274a592 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:01.615055 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1f0d4-cc44-4281-bd5b-26aa6274a592 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:01.615358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1f0d4-cc44-4281-bd5b-26aa6274a592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:01.646162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0ab1f0d4-cc44-4281-bd5b-26aa6274a592 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:01.656825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0ab1f0d4-cc44-4281-bd5b-26aa6274a592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:01.670616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:01.680989 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:01.704586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca885ff-d41b-42f5-9944-c1a18341facf None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:01.743637 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a773510-c0e4-4870-873f-2243e9c65932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:01.744941 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a773510-c0e4-4870-873f-2243e9c65932 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:01.745382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a773510-c0e4-4870-873f-2243e9c65932 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:01.745795 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a773510-c0e4-4870-873f-2243e9c65932 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:01.746649 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a773510-c0e4-4870-873f-2243e9c65932 None admin] RBAC: Adding query filter params (role.id=3acdf08298c84325b53a0e286cc53b30, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=b629ad3e6b9f4723b103f9845df2aaaf) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:01.747198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a773510-c0e4-4870-873f-2243e9c65932 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:01.748785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a773510-c0e4-4870-873f-2243e9c65932 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:01.755887 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 340/693] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:01 2022] GET /identity/v3/role_assignments?user.id=b629ad3e6b9f4723b103f9845df2aaaf&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=3acdf08298c84325b53a0e286cc53b30 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:01.799778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 354/694] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:33:01.950379 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a773510-c0e4-4870-873f-2243e9c65932 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:02.067223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a773510-c0e4-4870-873f-2243e9c65932 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:02.108628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c56a1e-8853-4398-8ac4-e93aa71c88e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:02.109314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c56a1e-8853-4398-8ac4-e93aa71c88e1 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:02.109592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c56a1e-8853-4398-8ac4-e93aa71c88e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:02.109864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c56a1e-8853-4398-8ac4-e93aa71c88e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:02.113713 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:02.121083 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c56a1e-8853-4398-8ac4-e93aa71c88e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-f4c56a1e-8853-4398-8ac4-e93aa71c88e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:02.125910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:02.126822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:02.127630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:02.127630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:02.127630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:02.127630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:02.127630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:02.127630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:02.127630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:02.127630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:02.127630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:02.127630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:02.127630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:02.127630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:02.127630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 341/695] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:33:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:02.133238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1f0d4-cc44-4281-bd5b-26aa6274a592 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:02.183107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c56a1e-8853-4398-8ac4-e93aa71c88e1 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:02.240519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1f0d4-cc44-4281-bd5b-26aa6274a592 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:02.279975 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9cc9b-853c-4fd4-b265-4feee7dc401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:02.280219 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9cc9b-853c-4fd4-b265-4feee7dc401d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:02.280621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9cc9b-853c-4fd4-b265-4feee7dc401d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:02.280745 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9cc9b-853c-4fd4-b265-4feee7dc401d None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:02.281463 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9cc9b-853c-4fd4-b265-4feee7dc401d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:02.281786 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9cc9b-853c-4fd4-b265-4feee7dc401d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:02.283244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9cc9b-853c-4fd4-b265-4feee7dc401d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:02.296779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 355/696] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:33:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:02.300349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9cc9b-853c-4fd4-b265-4feee7dc401d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:02.303367 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c56a1e-8853-4398-8ac4-e93aa71c88e1 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:02.304670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:02.343059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc76f30-c77d-420f-acc7-084a2c4471c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:02.344036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc76f30-c77d-420f-acc7-084a2c4471c5 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:02.344539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc76f30-c77d-420f-acc7-084a2c4471c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:02.345052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc76f30-c77d-420f-acc7-084a2c4471c5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:02.345827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc76f30-c77d-420f-acc7-084a2c4471c5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:02.347391 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc76f30-c77d-420f-acc7-084a2c4471c5 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-4fc76f30-c77d-420f-acc7-084a2c4471c5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:02.359977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:02.361097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:02.361934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:02.361934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:02.361934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:02.361934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:02.361934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:02.361934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:02.361934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:02.361934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:02.361934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:02.361934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:02.361934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:02.361934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:02.361934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:33:02.361934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:02.365005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 342/697] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Aug 24 07:33:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:02.370884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc76f30-c77d-420f-acc7-084a2c4471c5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:02.414883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9cc9b-853c-4fd4-b265-4feee7dc401d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:02.450646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:02.451669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e9afe-3b8d-4405-bf5a-8c90c68c84dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:02.452475 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e9afe-3b8d-4405-bf5a-8c90c68c84dc None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:02.453006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e9afe-3b8d-4405-bf5a-8c90c68c84dc None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:02.453409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e9afe-3b8d-4405-bf5a-8c90c68c84dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:02.465669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e9afe-3b8d-4405-bf5a-8c90c68c84dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:02.467446 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e9afe-3b8d-4405-bf5a-8c90c68c84dc None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-fa7e9afe-3b8d-4405-bf5a-8c90c68c84dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:02.484178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:02.485383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:02.486541 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:02.486541 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:02.486541 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:02.493238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 356/698] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:02.495158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e9afe-3b8d-4405-bf5a-8c90c68c84dc None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:02.564737 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc76f30-c77d-420f-acc7-084a2c4471c5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:02.577070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:02.614737 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e9afe-3b8d-4405-bf5a-8c90c68c84dc None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:02.637912 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26879c02-f803-494e-8c70-fc34f2d7306f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:02.638692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26879c02-f803-494e-8c70-fc34f2d7306f None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:02.639070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26879c02-f803-494e-8c70-fc34f2d7306f None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:02.639439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26879c02-f803-494e-8c70-fc34f2d7306f None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:02.640170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26879c02-f803-494e-8c70-fc34f2d7306f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:02.640572 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26879c02-f803-494e-8c70-fc34f2d7306f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:02.642220 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26879c02-f803-494e-8c70-fc34f2d7306f None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:02.648098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 343/699] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Aug 24 07:33:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:02.651343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beeb01f-f366-44a5-8132-3a797a980977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:02.652223 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beeb01f-f366-44a5-8132-3a797a980977 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:02.652639 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beeb01f-f366-44a5-8132-3a797a980977 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:02.653104 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beeb01f-f366-44a5-8132-3a797a980977 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:02.653722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26879c02-f803-494e-8c70-fc34f2d7306f None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:02.654067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beeb01f-f366-44a5-8132-3a797a980977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:02.654515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beeb01f-f366-44a5-8132-3a797a980977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:02.656040 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beeb01f-f366-44a5-8132-3a797a980977 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:02.672765 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 357/700] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:02.675408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beeb01f-f366-44a5-8132-3a797a980977 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:02.770087 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26879c02-f803-494e-8c70-fc34f2d7306f None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:02.805098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:02.834505 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a5bb7-cfc6-4a53-ba00-974d02564975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:02.835438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a5bb7-cfc6-4a53-ba00-974d02564975 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:02.835892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a5bb7-cfc6-4a53-ba00-974d02564975 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:02.836346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a5bb7-cfc6-4a53-ba00-974d02564975 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:02.837174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a5bb7-cfc6-4a53-ba00-974d02564975 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:02.837651 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a5bb7-cfc6-4a53-ba00-974d02564975 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:02.839216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a5bb7-cfc6-4a53-ba00-974d02564975 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:02.842161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beeb01f-f366-44a5-8132-3a797a980977 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:02.857314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 344/701] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Aug 24 07:33:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:02.869452 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:02.887553 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766ff70-ce44-479f-879b-d5e34675ed48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:02.888306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766ff70-ce44-479f-879b-d5e34675ed48 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:02.896246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766ff70-ce44-479f-879b-d5e34675ed48 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:02.896246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766ff70-ce44-479f-879b-d5e34675ed48 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:02.898617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766ff70-ce44-479f-879b-d5e34675ed48 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:02.900244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766ff70-ce44-479f-879b-d5e34675ed48 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-8766ff70-ce44-479f-879b-d5e34675ed48 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:02.915459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:02.916685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:02.917999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:02.917999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:02.917999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:02.917999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:02.917999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:02.917999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:02.917999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:02.917999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:02.917999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:33:02.917999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:02.923180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 358/702] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 24 07:33:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:02.924670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a5bb7-cfc6-4a53-ba00-974d02564975 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:03.068727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a5bb7-cfc6-4a53-ba00-974d02564975 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:03.105353 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a39724-3f87-4622-8914-947ca9b5ee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:03.105976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a39724-3f87-4622-8914-947ca9b5ee27 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:03.106250 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a39724-3f87-4622-8914-947ca9b5ee27 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:03.106519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a39724-3f87-4622-8914-947ca9b5ee27 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:03.107268 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a39724-3f87-4622-8914-947ca9b5ee27 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:03.107601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a39724-3f87-4622-8914-947ca9b5ee27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:03.109125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a39724-3f87-4622-8914-947ca9b5ee27 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:03.116386 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66b244-ed83-48fe-8212-e14fe8b4072b None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:03.117033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66b244-ed83-48fe-8212-e14fe8b4072b None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:03.117524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66b244-ed83-48fe-8212-e14fe8b4072b None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:03.119105 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 359/703] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:03.124709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55da9d-3a21-4a14-853e-1e4b99caf5a0 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:03.125273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55da9d-3a21-4a14-853e-1e4b99caf5a0 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:03.125771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55da9d-3a21-4a14-853e-1e4b99caf5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:03.130176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 345/704] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 24 07:33:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:03.136670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a39724-3f87-4622-8914-947ca9b5ee27 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:03.154348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-be55da9d-3a21-4a14-853e-1e4b99caf5a0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:03.155948 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-be55da9d-3a21-4a14-853e-1e4b99caf5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:03.187975 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:03.233004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 360/705] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:33:03.240054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c7a2c-89be-4847-8299-11fa83b1b5aa None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:03.240343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c7a2c-89be-4847-8299-11fa83b1b5aa None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:03.240665 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c7a2c-89be-4847-8299-11fa83b1b5aa None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:03.242268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 361/706] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:03.246860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d256a-cdfb-40b8-9b8d-724240451d7a None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:03.247143 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d256a-cdfb-40b8-9b8d-724240451d7a None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:03.247414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d256a-cdfb-40b8-9b8d-724240451d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:03.254349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a39724-3f87-4622-8914-947ca9b5ee27 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:03.272236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-051d256a-cdfb-40b8-9b8d-724240451d7a None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:03.273450 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-051d256a-cdfb-40b8-9b8d-724240451d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:03.310929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5499f4-7376-4732-80a8-0ecc79824629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:03.311539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5499f4-7376-4732-80a8-0ecc79824629 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:03.311820 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5499f4-7376-4732-80a8-0ecc79824629 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:03.312096 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5499f4-7376-4732-80a8-0ecc79824629 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:03.349030 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 362/707] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:33:03.422720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5499f4-7376-4732-80a8-0ecc79824629 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:03.424182 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5499f4-7376-4732-80a8-0ecc79824629 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:03.492682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:03.565902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 346/708] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:33:03.573743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d256a-cdfb-40b8-9b8d-724240451d7a None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:03.604994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5499f4-7376-4732-80a8-0ecc79824629 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:03.627422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:03.746563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d256a-cdfb-40b8-9b8d-724240451d7a None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:03.747690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5499f4-7376-4732-80a8-0ecc79824629 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:03.789673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b618c09-a032-43e1-aeb4-83926292ae50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:03.790316 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b618c09-a032-43e1-aeb4-83926292ae50 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:03.790601 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b618c09-a032-43e1-aeb4-83926292ae50 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:03.790877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b618c09-a032-43e1-aeb4-83926292ae50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:03.792850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5245e18-0438-4bdc-9d66-130e80583d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:03.793952 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5245e18-0438-4bdc-9d66-130e80583d99 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:03.794307 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5245e18-0438-4bdc-9d66-130e80583d99 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:03.794666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5245e18-0438-4bdc-9d66-130e80583d99 None admin] PATH_INFO: `/v3/services/network` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:03.795327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5245e18-0438-4bdc-9d66-130e80583d99 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:03.796798 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5245e18-0438-4bdc-9d66-130e80583d99 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:03.802401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b618c09-a032-43e1-aeb4-83926292ae50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:03.803851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b618c09-a032-43e1-aeb4-83926292ae50 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-e5245e18-0438-4bdc-9d66-130e80583d99 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:03.808118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:03.809125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:03.809964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 347/709] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:03 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:03.812163 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5245e18-0438-4bdc-9d66-130e80583d99 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-0b618c09-a032-43e1-aeb4-83926292ae50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:03.822468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:03.824556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:03.826645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:03.826645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:03.826645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:03.829022 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:03.829208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 363/710] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:03.830449 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b618c09-a032-43e1-aeb4-83926292ae50 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:03.934164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:03.939809 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5245e18-0438-4bdc-9d66-130e80583d99 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:03.947657 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b618c09-a032-43e1-aeb4-83926292ae50 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:03.985744 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722bab6-072d-42f7-b23c-b183b09fd754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:03.986378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722bab6-072d-42f7-b23c-b183b09fd754 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:03.986659 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722bab6-072d-42f7-b23c-b183b09fd754 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:03.986932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722bab6-072d-42f7-b23c-b183b09fd754 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:03.988103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa3f5c-9350-49f7-882a-5ca01d07e0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:03.988737 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa3f5c-9350-49f7-882a-5ca01d07e0e4 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:03.989124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa3f5c-9350-49f7-882a-5ca01d07e0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:03.989416 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa3f5c-9350-49f7-882a-5ca01d07e0e4 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:03.990036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa3f5c-9350-49f7-882a-5ca01d07e0e4 None admin] RBAC: Adding query filter params (name=network) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:03.990323 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa3f5c-9350-49f7-882a-5ca01d07e0e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:03.992394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa3f5c-9350-49f7-882a-5ca01d07e0e4 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:04.001420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 364/711] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 24 07:33:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:04.001420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa3f5c-9350-49f7-882a-5ca01d07e0e4 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:04.006696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722bab6-072d-42f7-b23c-b183b09fd754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:04.010008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722bab6-072d-42f7-b23c-b183b09fd754 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-e722bab6-072d-42f7-b23c-b183b09fd754 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:04.027500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:04.028556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:04.029529 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:04.029529 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:04.029529 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:04.032720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 348/712] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:04.033790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722bab6-072d-42f7-b23c-b183b09fd754 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:04.112190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:04.123510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa3f5c-9350-49f7-882a-5ca01d07e0e4 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:04.163265 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bbfa6-88f5-445b-a1e0-80b5ae619a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:04.164283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bbfa6-88f5-445b-a1e0-80b5ae619a42 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:04.164394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722bab6-072d-42f7-b23c-b183b09fd754 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:04.164896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bbfa6-88f5-445b-a1e0-80b5ae619a42 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:04.165399 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bbfa6-88f5-445b-a1e0-80b5ae619a42 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:04.166335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bbfa6-88f5-445b-a1e0-80b5ae619a42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:04.166884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bbfa6-88f5-445b-a1e0-80b5ae619a42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:04.168453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bbfa6-88f5-445b-a1e0-80b5ae619a42 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:04.186358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 365/713] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:04.187673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bbfa6-88f5-445b-a1e0-80b5ae619a42 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:04.203936 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ae727-28c5-4b87-87a3-e3921a18a96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:04.204550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ae727-28c5-4b87-87a3-e3921a18a96a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:04.204862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ae727-28c5-4b87-87a3-e3921a18a96a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:04.205140 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ae727-28c5-4b87-87a3-e3921a18a96a None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:04.205774 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ae727-28c5-4b87-87a3-e3921a18a96a None admin] RBAC: Adding query filter params (type=network) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:04.206080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ae727-28c5-4b87-87a3-e3921a18a96a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:04.206672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:04.207732 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ae727-28c5-4b87-87a3-e3921a18a96a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:04.220407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 349/714] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 24 07:33:04 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:04.220407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ae727-28c5-4b87-87a3-e3921a18a96a None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:04.337552 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ae727-28c5-4b87-87a3-e3921a18a96a None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:04.385766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:04.387560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bbfa6-88f5-445b-a1e0-80b5ae619a42 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:04.410039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41f8f8-a5ba-4d10-939e-052975a23133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:04.411387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41f8f8-a5ba-4d10-939e-052975a23133 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:04.411893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41f8f8-a5ba-4d10-939e-052975a23133 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:04.412369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41f8f8-a5ba-4d10-939e-052975a23133 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:04.429109 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0d765-3bd1-4d88-bccb-8781017cbd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:04.429736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0d765-3bd1-4d88-bccb-8781017cbd97 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:04.430020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0d765-3bd1-4d88-bccb-8781017cbd97 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:04.430297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0d765-3bd1-4d88-bccb-8781017cbd97 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:04.430860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41f8f8-a5ba-4d10-939e-052975a23133 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:04.431033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0d765-3bd1-4d88-bccb-8781017cbd97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:04.431313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0d765-3bd1-4d88-bccb-8781017cbd97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:04.432762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0d765-3bd1-4d88-bccb-8781017cbd97 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:04.433118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41f8f8-a5ba-4d10-939e-052975a23133 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:04.447828 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 366/715] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:04.453746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0d765-3bd1-4d88-bccb-8781017cbd97 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-3f41f8f8-a5ba-4d10-939e-052975a23133 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:04.458563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:04.459446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:04.460313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:04.460313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:04.460313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:04.460313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:04.460313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:04.460313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:04.460313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:04.460313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:04.467313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 350/716] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:04.468815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41f8f8-a5ba-4d10-939e-052975a23133 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:04.544888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:04.592299 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0d765-3bd1-4d88-bccb-8781017cbd97 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:04.601655 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41f8f8-a5ba-4d10-939e-052975a23133 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:04.637740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11ec24-2086-46ad-afb2-2afcb5933dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:04.638743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ec24-2086-46ad-afb2-2afcb5933dbe None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:04.639252 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ec24-2086-46ad-afb2-2afcb5933dbe None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:04.639818 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ec24-2086-46ad-afb2-2afcb5933dbe None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:04.640848 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11ec24-2086-46ad-afb2-2afcb5933dbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:04.642594 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea6d25-3612-41a0-a08f-51f786f9e5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:04.642911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11ec24-2086-46ad-afb2-2afcb5933dbe None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:04.643352 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea6d25-3612-41a0-a08f-51f786f9e5d3 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:04.643704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea6d25-3612-41a0-a08f-51f786f9e5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:04.644049 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea6d25-3612-41a0-a08f-51f786f9e5d3 None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:04.644874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea6d25-3612-41a0-a08f-51f786f9e5d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:04.645244 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea6d25-3612-41a0-a08f-51f786f9e5d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:04.646733 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea6d25-3612-41a0-a08f-51f786f9e5d3 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:04.665131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 351/717] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:04.665131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ec9d3-492d-4a1d-b10d-2d500fc252f6 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:04.665131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ec9d3-492d-4a1d-b10d-2d500fc252f6 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:04.665131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ec9d3-492d-4a1d-b10d-2d500fc252f6 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:04.668228 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 352/718] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:04.669027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ec9d3-492d-4a1d-b10d-2d500fc252f6 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:04.719176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:04.743585 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 367/719] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:33:04 2022] POST /identity/v3/users => generated 254 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:33:04.745924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b706ad0-63cb-42fc-b798-30578aeca068 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:04.746512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b706ad0-63cb-42fc-b798-30578aeca068 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:04.747008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b706ad0-63cb-42fc-b798-30578aeca068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:04.770537 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9b706ad0-63cb-42fc-b798-30578aeca068 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:04.773026 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9b706ad0-63cb-42fc-b798-30578aeca068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:04.842420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ec9d3-492d-4a1d-b10d-2d500fc252f6 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:04.866700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 368/720] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:33:04.904169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91800c-9f7e-4342-84aa-ff02aca703cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:04.904902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91800c-9f7e-4342-84aa-ff02aca703cb None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:04.905179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91800c-9f7e-4342-84aa-ff02aca703cb None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:04.905450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91800c-9f7e-4342-84aa-ff02aca703cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:04.906181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91800c-9f7e-4342-84aa-ff02aca703cb None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=bdf81f09006a42299c0807f77f16c748) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:04.906493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91800c-9f7e-4342-84aa-ff02aca703cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:04.907923 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91800c-9f7e-4342-84aa-ff02aca703cb None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:04.915176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 353/721] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:04 2022] GET /identity/v3/role_assignments?user.id=bdf81f09006a42299c0807f77f16c748&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:05.159060 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b706ad0-63cb-42fc-b798-30578aeca068 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:05.191626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219dfd2-74d3-45cc-b525-509fc25da675 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:05.191904 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219dfd2-74d3-45cc-b525-509fc25da675 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:05.192246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219dfd2-74d3-45cc-b525-509fc25da675 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:05.194060 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 354/722] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:05.199133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099a38c-a376-4a6a-93c2-7aa3a1bae64f None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:05.199427 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099a38c-a376-4a6a-93c2-7aa3a1bae64f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:05.199704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099a38c-a376-4a6a-93c2-7aa3a1bae64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:05.229133 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:05.230552 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-6099a38c-a376-4a6a-93c2-7aa3a1bae64f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:05.231759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6099a38c-a376-4a6a-93c2-7aa3a1bae64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:05.276091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b706ad0-63cb-42fc-b798-30578aeca068 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:05.311542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 355/723] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:33:05.318930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ccedf-1fe8-4077-8358-fa6a2454d950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:05.319568 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ccedf-1fe8-4077-8358-fa6a2454d950 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:05.319851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ccedf-1fe8-4077-8358-fa6a2454d950 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:05.320170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ccedf-1fe8-4077-8358-fa6a2454d950 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:05.330417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ccedf-1fe8-4077-8358-fa6a2454d950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-946ccedf-1fe8-4077-8358-fa6a2454d950 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:05.335350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:05.336244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:05.337215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:05.337215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:05.337215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:05.337215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:05.337215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:05.337215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:05.337215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:05.337215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:05.337215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:05.337215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:05.337215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:05.337215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:05.337215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 369/724] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:33:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:05.344807 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099a38c-a376-4a6a-93c2-7aa3a1bae64f None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:05.413637 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:05.489729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099a38c-a376-4a6a-93c2-7aa3a1bae64f None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:05.534770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa98645-e9e8-4b84-9f9a-8b04dc070ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:05.535562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa98645-e9e8-4b84-9f9a-8b04dc070ddf None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:05.536041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa98645-e9e8-4b84-9f9a-8b04dc070ddf None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:05.536323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa98645-e9e8-4b84-9f9a-8b04dc070ddf None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:05.537076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa98645-e9e8-4b84-9f9a-8b04dc070ddf None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:05.537384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa98645-e9e8-4b84-9f9a-8b04dc070ddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:05.538840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa98645-e9e8-4b84-9f9a-8b04dc070ddf None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:05.548036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 356/725] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:33:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:05.554635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ccedf-1fe8-4077-8358-fa6a2454d950 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:05.651137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa98645-e9e8-4b84-9f9a-8b04dc070ddf None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:05.711801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ccedf-1fe8-4077-8358-fa6a2454d950 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:05.730437 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:05.750239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b392d-99a7-4402-9061-c28eb909565c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:05.750921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b392d-99a7-4402-9061-c28eb909565c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:05.751219 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b392d-99a7-4402-9061-c28eb909565c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:05.751493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b392d-99a7-4402-9061-c28eb909565c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:05.764936 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b392d-99a7-4402-9061-c28eb909565c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:05.766483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b392d-99a7-4402-9061-c28eb909565c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-7b1b392d-99a7-4402-9061-c28eb909565c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:05.781306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:05.782238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:05.783116 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:05.783116 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:05.783116 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:05.783116 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 370/726] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:05.789455 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b392d-99a7-4402-9061-c28eb909565c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:05.852776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:05.856178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa98645-e9e8-4b84-9f9a-8b04dc070ddf None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:05.898094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b392d-99a7-4402-9061-c28eb909565c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:05.906055 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d56704-25cf-4c7c-917e-962829856377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:05.906671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d56704-25cf-4c7c-917e-962829856377 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:05.906958 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d56704-25cf-4c7c-917e-962829856377 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:05.907237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d56704-25cf-4c7c-917e-962829856377 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:05.907887 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d56704-25cf-4c7c-917e-962829856377 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:05.909290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d56704-25cf-4c7c-917e-962829856377 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:05.937502 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13c905-8afb-4d06-8192-1d5f6a9c139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:05.938168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13c905-8afb-4d06-8192-1d5f6a9c139b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:05.938451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13c905-8afb-4d06-8192-1d5f6a9c139b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:05.938726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13c905-8afb-4d06-8192-1d5f6a9c139b None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:05.939420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13c905-8afb-4d06-8192-1d5f6a9c139b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:05.939741 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13c905-8afb-4d06-8192-1d5f6a9c139b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:05.941954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13c905-8afb-4d06-8192-1d5f6a9c139b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:05.954708 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 371/727] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:05.960714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13c905-8afb-4d06-8192-1d5f6a9c139b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:05.990439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 357/728] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 24 07:33:05 2022] POST /identity/v3/services => generated 251 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:33:06.029074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:06.076592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13c905-8afb-4d06-8192-1d5f6a9c139b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:06.123549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b18d05-c355-4d57-bacf-169a39f8778b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:06.124203 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b18d05-c355-4d57-bacf-169a39f8778b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:06.124489 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b18d05-c355-4d57-bacf-169a39f8778b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:06.124804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b18d05-c355-4d57-bacf-169a39f8778b None admin] PATH_INFO: `/v3/users/swift` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:06.133562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b18d05-c355-4d57-bacf-169a39f8778b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:06.135030 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b18d05-c355-4d57-bacf-169a39f8778b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-e6b18d05-c355-4d57-bacf-169a39f8778b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:06.148634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:06.149694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:06.151074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:06.151074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:06.151074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:06.151074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:06.151074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:06.151074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:06.151074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:06.151074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:06.151074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:33:06.151074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:06.151074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 372/729] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:33:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:06.155968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d56704-25cf-4c7c-917e-962829856377 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:06.286967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d56704-25cf-4c7c-917e-962829856377 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:06.329138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f9146-7574-499d-ae4d-6db12a55df5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:06.330098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f9146-7574-499d-ae4d-6db12a55df5a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:06.330570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f9146-7574-499d-ae4d-6db12a55df5a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:06.331018 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f9146-7574-499d-ae4d-6db12a55df5a None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:06.332192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f9146-7574-499d-ae4d-6db12a55df5a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:06.332771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f9146-7574-499d-ae4d-6db12a55df5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:06.334463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f9146-7574-499d-ae4d-6db12a55df5a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:06.356704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 358/730] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Aug 24 07:33:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:06.361888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b18d05-c355-4d57-bacf-169a39f8778b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:06.470863 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:06.488908 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b18d05-c355-4d57-bacf-169a39f8778b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:06.537657 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f0de3-420b-460e-99f1-e3f74f984357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:06.538220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f0de3-420b-460e-99f1-e3f74f984357 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:06.538501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f0de3-420b-460e-99f1-e3f74f984357 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:06.538777 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f0de3-420b-460e-99f1-e3f74f984357 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:06.638692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f0de3-420b-460e-99f1-e3f74f984357 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:06.640153 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f0de3-420b-460e-99f1-e3f74f984357 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:06.747245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:06.753591 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb6443-c4f6-4947-8234-bbd715316ea0 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:06.754167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb6443-c4f6-4947-8234-bbd715316ea0 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:06.754448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb6443-c4f6-4947-8234-bbd715316ea0 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:06.756052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 359/731] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:06.761439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389ece6-0781-49df-bba6-cf0c10e8d2df None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:06.761965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389ece6-0781-49df-bba6-cf0c10e8d2df None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:06.766113 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389ece6-0781-49df-bba6-cf0c10e8d2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:06.775629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 373/732] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:06 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:33:06.784004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f0de3-420b-460e-99f1-e3f74f984357 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:06.801643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-5389ece6-0781-49df-bba6-cf0c10e8d2df None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:06.802768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5389ece6-0781-49df-bba6-cf0c10e8d2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:06.828284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:06.915262 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f0de3-420b-460e-99f1-e3f74f984357 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:06.957901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 360/733] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:33:06.984164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05956238-d1c4-43b1-9634-4bdce1febb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:06.985130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05956238-d1c4-43b1-9634-4bdce1febb9b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:06.989283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05956238-d1c4-43b1-9634-4bdce1febb9b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:06.989283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05956238-d1c4-43b1-9634-4bdce1febb9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:07.004877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05956238-d1c4-43b1-9634-4bdce1febb9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:07.006601 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05956238-d1c4-43b1-9634-4bdce1febb9b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-05956238-d1c4-43b1-9634-4bdce1febb9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:07.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:07.033506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:07.034596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:07.034596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:07.034596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:07.040445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 374/734] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:07.043358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389ece6-0781-49df-bba6-cf0c10e8d2df None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:07.167293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389ece6-0781-49df-bba6-cf0c10e8d2df None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:07.208946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a805ec-2c21-4ae2-9b7e-114c69572913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:07.209835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a805ec-2c21-4ae2-9b7e-114c69572913 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:07.210259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a805ec-2c21-4ae2-9b7e-114c69572913 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:07.210671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a805ec-2c21-4ae2-9b7e-114c69572913 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:07.211455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a805ec-2c21-4ae2-9b7e-114c69572913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:07.211914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a805ec-2c21-4ae2-9b7e-114c69572913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:07.213454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a805ec-2c21-4ae2-9b7e-114c69572913 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:07.226456 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 361/735] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:07.233700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05956238-d1c4-43b1-9634-4bdce1febb9b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:07.282130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:07.329299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528875a8-4ad9-4e61-bbf8-efffe699da69 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:07.329589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528875a8-4ad9-4e61-bbf8-efffe699da69 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:07.330124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528875a8-4ad9-4e61-bbf8-efffe699da69 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:07.332569 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 362/736] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:07.337671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8584cf2-abbc-4e17-9c5a-21ce77701010 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:07.338248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8584cf2-abbc-4e17-9c5a-21ce77701010 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:07.338737 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8584cf2-abbc-4e17-9c5a-21ce77701010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:07.357504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e8584cf2-abbc-4e17-9c5a-21ce77701010 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:07.359056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e8584cf2-abbc-4e17-9c5a-21ce77701010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:07.361461 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:07.371848 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05956238-d1c4-43b1-9634-4bdce1febb9b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:07.433458 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66576dce-77a9-4ece-92cf-ac2677441ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:07.434041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66576dce-77a9-4ece-92cf-ac2677441ac1 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:07.434328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66576dce-77a9-4ece-92cf-ac2677441ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:07.434605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66576dce-77a9-4ece-92cf-ac2677441ac1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:07.442119 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66576dce-77a9-4ece-92cf-ac2677441ac1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:07.443555 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66576dce-77a9-4ece-92cf-ac2677441ac1 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:07.454067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 363/737] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:33:07.454871 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8584cf2-abbc-4e17-9c5a-21ce77701010 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-66576dce-77a9-4ece-92cf-ac2677441ac1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:07.455284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:07.456434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:07.457650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:07.457650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:07.457650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:07.457650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:07.457650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:07.457650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:07.457650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:07.457650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:07.457650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 375/738] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:07.458168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacc950-f1d3-4b69-b0d3-a47293e82e76 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:07.460107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacc950-f1d3-4b69-b0d3-a47293e82e76 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:07.460107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacc950-f1d3-4b69-b0d3-a47293e82e76 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:07.460287 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 376/739] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:07.463145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacc950-f1d3-4b69-b0d3-a47293e82e76 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:07.587709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:07.592906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacc950-f1d3-4b69-b0d3-a47293e82e76 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:07.593931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8584cf2-abbc-4e17-9c5a-21ce77701010 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:07.629365 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e44567-1ae5-47f5-922c-2a724d658cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:07.630556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e44567-1ae5-47f5-922c-2a724d658cd8 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:07.631056 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e44567-1ae5-47f5-922c-2a724d658cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:07.631375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffc27f-da22-422c-8d70-b248698beb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:07.631691 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e44567-1ae5-47f5-922c-2a724d658cd8 None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:07.631980 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffc27f-da22-422c-8d70-b248698beb5b None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:07.632263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffc27f-da22-422c-8d70-b248698beb5b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:07.632539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffc27f-da22-422c-8d70-b248698beb5b None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:07.633044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e44567-1ae5-47f5-922c-2a724d658cd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:07.633253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffc27f-da22-422c-8d70-b248698beb5b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:07.633667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e44567-1ae5-47f5-922c-2a724d658cd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:07.634685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffc27f-da22-422c-8d70-b248698beb5b None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:07.635441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e44567-1ae5-47f5-922c-2a724d658cd8 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:07.647462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 377/740] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:07.648926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4211b0-c543-4806-b22c-c12162925917 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:07.649447 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4211b0-c543-4806-b22c-c12162925917 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:07.649911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4211b0-c543-4806-b22c-c12162925917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:07.669486 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-4b4211b0-c543-4806-b22c-c12162925917 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:07.670662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4b4211b0-c543-4806-b22c-c12162925917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:07.687107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:07.704965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 364/741] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 24 07:33:07 2022] POST /identity/v3/services => generated 240 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:33:07.706006 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffc27f-da22-422c-8d70-b248698beb5b None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:07.767558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 378/742] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:07.768474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4211b0-c543-4806-b22c-c12162925917 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:07.861732 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffc27f-da22-422c-8d70-b248698beb5b None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:07.877968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:07.898342 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4211b0-c543-4806-b22c-c12162925917 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:07.911392 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e60d0-b5e5-4058-acbb-3c03f27a4aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:07.912035 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e60d0-b5e5-4058-acbb-3c03f27a4aaa None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:07.912739 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e60d0-b5e5-4058-acbb-3c03f27a4aaa None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:07.913032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e60d0-b5e5-4058-acbb-3c03f27a4aaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:07.913779 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e60d0-b5e5-4058-acbb-3c03f27a4aaa None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=b629ad3e6b9f4723b103f9845df2aaaf) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:07.914371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e60d0-b5e5-4058-acbb-3c03f27a4aaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:07.916125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e60d0-b5e5-4058-acbb-3c03f27a4aaa None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:07.924335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 365/743] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:07 2022] GET /identity/v3/role_assignments?user.id=b629ad3e6b9f4723b103f9845df2aaaf&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:07.959919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d14d7-fbec-471e-ba8a-fb62da15a5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:07.960869 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d14d7-fbec-471e-ba8a-fb62da15a5e4 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:07.961317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d14d7-fbec-471e-ba8a-fb62da15a5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:07.961735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d14d7-fbec-471e-ba8a-fb62da15a5e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:07.974666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d14d7-fbec-471e-ba8a-fb62da15a5e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-180d14d7-fbec-471e-ba8a-fb62da15a5e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:07.979812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:07.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:07.990706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:07.990706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:07.990706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:07.990706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:07.990706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:07.990706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:07.990706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:07.990706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:07.990706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:07.990706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:07.990706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:07.990706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:07.990706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 379/744] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:33:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:07.992097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e60d0-b5e5-4058-acbb-3c03f27a4aaa None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:08.067252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:08.111510 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e60d0-b5e5-4058-acbb-3c03f27a4aaa None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:08.134962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d14d7-fbec-471e-ba8a-fb62da15a5e4 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:08.148242 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dc942-5c3c-45b1-a21c-c8e2714ab802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:08.149028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dc942-5c3c-45b1-a21c-c8e2714ab802 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:08.149411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dc942-5c3c-45b1-a21c-c8e2714ab802 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:08.149790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dc942-5c3c-45b1-a21c-c8e2714ab802 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:08.150553 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dc942-5c3c-45b1-a21c-c8e2714ab802 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:08.150961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dc942-5c3c-45b1-a21c-c8e2714ab802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:08.152433 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dc942-5c3c-45b1-a21c-c8e2714ab802 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:08.161529 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 366/745] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:33:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:08.167315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dc942-5c3c-45b1-a21c-c8e2714ab802 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:08.285066 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d14d7-fbec-471e-ba8a-fb62da15a5e4 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:08.319880 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dc942-5c3c-45b1-a21c-c8e2714ab802 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:08.323805 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069d04d-8532-4535-9326-3f3f71d5051e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:08.324613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069d04d-8532-4535-9326-3f3f71d5051e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:08.325016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069d04d-8532-4535-9326-3f3f71d5051e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:08.327119 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069d04d-8532-4535-9326-3f3f71d5051e None admin] PATH_INFO: `/v3/roles/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:08.350213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069d04d-8532-4535-9326-3f3f71d5051e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-7069d04d-8532-4535-9326-3f3f71d5051e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:08.359856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:08.360831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:08.361670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:08.361670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:08.361670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:08.361670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:08.361670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:08.361670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:08.361670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:08.361670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:08.361670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:08.361670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:08.361670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:33:08.361670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:08.361670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 380/746] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:33:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:08.366898 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069d04d-8532-4535-9326-3f3f71d5051e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:08.374401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:08.374963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:08.442451 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e1bc8-01db-4f42-a036-17e3cda8e885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:08.443101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e1bc8-01db-4f42-a036-17e3cda8e885 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:08.443384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e1bc8-01db-4f42-a036-17e3cda8e885 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:08.443658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e1bc8-01db-4f42-a036-17e3cda8e885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:08.468253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e1bc8-01db-4f42-a036-17e3cda8e885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:08.469763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e1bc8-01db-4f42-a036-17e3cda8e885 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-7a2e1bc8-01db-4f42-a036-17e3cda8e885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:08.492410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:08.493631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:08.494539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:08.494539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:08.494539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:08.495551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 367/747] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:08.496761 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b984d-e615-469b-aaeb-176d592be881 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:08.497048 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b984d-e615-469b-aaeb-176d592be881 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:08.497386 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b984d-e615-469b-aaeb-176d592be881 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:08.498877 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 368/748] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:08.504571 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d7df0-a011-40a0-bf20-bdaca0a3b992 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:08.504931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d7df0-a011-40a0-bf20-bdaca0a3b992 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:08.505265 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d7df0-a011-40a0-bf20-bdaca0a3b992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:08.512510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069d04d-8532-4535-9326-3f3f71d5051e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:08.523657 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e28d7df0-a011-40a0-bf20-bdaca0a3b992 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:08.524897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e28d7df0-a011-40a0-bf20-bdaca0a3b992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:08.553412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b1493-aa80-4b31-bca4-c5cce62753e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:08.554183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b1493-aa80-4b31-bca4-c5cce62753e2 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:08.554471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b1493-aa80-4b31-bca4-c5cce62753e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:08.555070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b1493-aa80-4b31-bca4-c5cce62753e2 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:08.555901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b1493-aa80-4b31-bca4-c5cce62753e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:08.556334 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b1493-aa80-4b31-bca4-c5cce62753e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:08.557913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b1493-aa80-4b31-bca4-c5cce62753e2 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:08.571641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 381/749] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:33:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:08.576961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b1493-aa80-4b31-bca4-c5cce62753e2 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:08.603418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 369/750] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:08.606043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d7df0-a011-40a0-bf20-bdaca0a3b992 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:08.661106 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:08.695606 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b1493-aa80-4b31-bca4-c5cce62753e2 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:08.761475 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4797f-79b0-4779-a1d3-21914bf5e368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:08.762085 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4797f-79b0-4779-a1d3-21914bf5e368 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:08.762369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4797f-79b0-4779-a1d3-21914bf5e368 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:08.762644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4797f-79b0-4779-a1d3-21914bf5e368 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:08.763357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4797f-79b0-4779-a1d3-21914bf5e368 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:08.763675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4797f-79b0-4779-a1d3-21914bf5e368 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:08.765082 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4797f-79b0-4779-a1d3-21914bf5e368 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:08.792258 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 382/751] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:08.793171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4797f-79b0-4779-a1d3-21914bf5e368 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:08.815247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d7df0-a011-40a0-bf20-bdaca0a3b992 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:08.828959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:08.878826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28ac96-f56c-4bd7-b4c4-cbde47ad4752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:08.879569 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28ac96-f56c-4bd7-b4c4-cbde47ad4752 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:08.879853 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28ac96-f56c-4bd7-b4c4-cbde47ad4752 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:08.880131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28ac96-f56c-4bd7-b4c4-cbde47ad4752 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:08.897649 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28ac96-f56c-4bd7-b4c4-cbde47ad4752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:08.899101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28ac96-f56c-4bd7-b4c4-cbde47ad4752 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-de28ac96-f56c-4bd7-b4c4-cbde47ad4752 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:08.921783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:08.922965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:08.924062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:08.924062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:08.924062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:08.924062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 370/752] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:08.924062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28ac96-f56c-4bd7-b4c4-cbde47ad4752 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:08.957504 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4797f-79b0-4779-a1d3-21914bf5e368 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:08.985777 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:08.996901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc83ce-53dc-4707-8b88-e1bb88b8b734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:08.997542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc83ce-53dc-4707-8b88-e1bb88b8b734 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:08.997883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc83ce-53dc-4707-8b88-e1bb88b8b734 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:08.998162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc83ce-53dc-4707-8b88-e1bb88b8b734 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:09.007734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc83ce-53dc-4707-8b88-e1bb88b8b734 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:09.009232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc83ce-53dc-4707-8b88-e1bb88b8b734 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-8ccc83ce-53dc-4707-8b88-e1bb88b8b734 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:09.025919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:09.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:09.027817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:09.027817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:09.027817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:09.027817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:09.027817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:09.027817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:09.027817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:09.027817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:09.027817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:33:09.027817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:09.031972 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 383/753] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 24 07:33:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:09.033015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc83ce-53dc-4707-8b88-e1bb88b8b734 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:09.043378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28ac96-f56c-4bd7-b4c4-cbde47ad4752 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:09.090937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d62b3d-9919-49ea-93c3-a515a79467c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:09.091645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d62b3d-9919-49ea-93c3-a515a79467c8 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:09.092045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d62b3d-9919-49ea-93c3-a515a79467c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:09.092330 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d62b3d-9919-49ea-93c3-a515a79467c8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:09.093184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d62b3d-9919-49ea-93c3-a515a79467c8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:09.095375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d62b3d-9919-49ea-93c3-a515a79467c8 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-59d62b3d-9919-49ea-93c3-a515a79467c8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:09.108012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:09.109705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:09.111076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 371/754] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Aug 24 07:33:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:09.112155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d62b3d-9919-49ea-93c3-a515a79467c8 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:09.146571 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:09.146824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc83ce-53dc-4707-8b88-e1bb88b8b734 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:09.190724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7d7da-874e-45f8-942b-f7548dd004ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:09.191881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7d7da-874e-45f8-942b-f7548dd004ff None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:09.192157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7d7da-874e-45f8-942b-f7548dd004ff None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:09.192422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7d7da-874e-45f8-942b-f7548dd004ff None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:09.193200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7d7da-874e-45f8-942b-f7548dd004ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:09.193505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7d7da-874e-45f8-942b-f7548dd004ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:09.194911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7d7da-874e-45f8-942b-f7548dd004ff None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:09.213575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 384/755] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:09.213575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7d7da-874e-45f8-942b-f7548dd004ff None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:09.250590 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d62b3d-9919-49ea-93c3-a515a79467c8 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:09.275127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:09.287675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cef4f-584f-4ea0-91b7-9e69868b7a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:09.290296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cef4f-584f-4ea0-91b7-9e69868b7a8c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:09.290296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cef4f-584f-4ea0-91b7-9e69868b7a8c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:09.290296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cef4f-584f-4ea0-91b7-9e69868b7a8c None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:09.290296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cef4f-584f-4ea0-91b7-9e69868b7a8c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:09.290296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cef4f-584f-4ea0-91b7-9e69868b7a8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:09.291313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cef4f-584f-4ea0-91b7-9e69868b7a8c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:09.310357 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 372/756] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 24 07:33:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:09.311202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cef4f-584f-4ea0-91b7-9e69868b7a8c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:09.353458 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7d7da-874e-45f8-942b-f7548dd004ff None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:09.394799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190d4b3-d1c1-4993-a92f-128ff2e98759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:09.395442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190d4b3-d1c1-4993-a92f-128ff2e98759 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:09.395718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190d4b3-d1c1-4993-a92f-128ff2e98759 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:09.395985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190d4b3-d1c1-4993-a92f-128ff2e98759 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:09.396614 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190d4b3-d1c1-4993-a92f-128ff2e98759 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:09.396941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190d4b3-d1c1-4993-a92f-128ff2e98759 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:09.398299 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190d4b3-d1c1-4993-a92f-128ff2e98759 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:09.403592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 385/757] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Aug 24 07:33:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:09.404548 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190d4b3-d1c1-4993-a92f-128ff2e98759 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:09.407835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:09.435890 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cef4f-584f-4ea0-91b7-9e69868b7a8c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:09.484154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b0390-4896-44ad-9e66-30b760695e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:09.484995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b0390-4896-44ad-9e66-30b760695e1f None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:09.485885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b0390-4896-44ad-9e66-30b760695e1f None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:09.485885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b0390-4896-44ad-9e66-30b760695e1f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:09.493498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b0390-4896-44ad-9e66-30b760695e1f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:09.495310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b0390-4896-44ad-9e66-30b760695e1f None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-737b0390-4896-44ad-9e66-30b760695e1f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:09.507267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:09.508396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:09.509582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:09.509582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:09.509582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:09.509582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:09.509582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:09.509582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:09.509582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:09.509582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:09.509582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:33:09.509582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:09.515999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 373/758] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 24 07:33:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:09.515999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b0390-4896-44ad-9e66-30b760695e1f None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:09.540572 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190d4b3-d1c1-4993-a92f-128ff2e98759 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:09.552631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:09.577167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d602c-c503-485c-b5c3-6781182e40d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:09.577740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d602c-c503-485c-b5c3-6781182e40d3 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:09.578028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d602c-c503-485c-b5c3-6781182e40d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:09.578300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d602c-c503-485c-b5c3-6781182e40d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:09.591976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d602c-c503-485c-b5c3-6781182e40d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:09.595568 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d602c-c503-485c-b5c3-6781182e40d3 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-eb1d602c-c503-485c-b5c3-6781182e40d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:09.620754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:09.621929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:09.623076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:09.623076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:09.623076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:09.629549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 386/759] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:09.631528 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d602c-c503-485c-b5c3-6781182e40d3 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:09.656727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b0390-4896-44ad-9e66-30b760695e1f None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:09.693929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792f0f2-8b6f-4806-999b-591e304171e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:09.694533 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792f0f2-8b6f-4806-999b-591e304171e0 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:09.694810 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792f0f2-8b6f-4806-999b-591e304171e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:09.695083 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792f0f2-8b6f-4806-999b-591e304171e0 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:09.695842 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792f0f2-8b6f-4806-999b-591e304171e0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:09.696061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792f0f2-8b6f-4806-999b-591e304171e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:09.697503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792f0f2-8b6f-4806-999b-591e304171e0 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:09.703044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 374/760] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Aug 24 07:33:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:09.703903 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792f0f2-8b6f-4806-999b-591e304171e0 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:09.707882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:09.761551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d602c-c503-485c-b5c3-6781182e40d3 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:09.799228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de815b0-9d75-43a0-a31d-11174ba20c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:09.801061 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de815b0-9d75-43a0-a31d-11174ba20c71 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:09.801661 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de815b0-9d75-43a0-a31d-11174ba20c71 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:09.802141 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de815b0-9d75-43a0-a31d-11174ba20c71 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:09.803251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de815b0-9d75-43a0-a31d-11174ba20c71 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:09.803796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de815b0-9d75-43a0-a31d-11174ba20c71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:09.805547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de815b0-9d75-43a0-a31d-11174ba20c71 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:09.809112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792f0f2-8b6f-4806-999b-591e304171e0 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:09.826783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 387/761] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Aug 24 07:33:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:09.828081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de815b0-9d75-43a0-a31d-11174ba20c71 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:09.844492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d254e-1694-42a7-be88-ac71ef526ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:09.845171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d254e-1694-42a7-be88-ac71ef526ecd None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:09.845486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d254e-1694-42a7-be88-ac71ef526ecd None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:09.845798 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d254e-1694-42a7-be88-ac71ef526ecd None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:09.846584 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d254e-1694-42a7-be88-ac71ef526ecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:09.846779 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d254e-1694-42a7-be88-ac71ef526ecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:09.848613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d254e-1694-42a7-be88-ac71ef526ecd None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:09.854445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:09.862980 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 375/762] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:09.863857 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d254e-1694-42a7-be88-ac71ef526ecd None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:09.933502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:09.943964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de815b0-9d75-43a0-a31d-11174ba20c71 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:09.974934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d254e-1694-42a7-be88-ac71ef526ecd None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:09.981725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb78cf-381a-43c2-9745-4c6e136dfb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:09.982418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb78cf-381a-43c2-9745-4c6e136dfb8a None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:09.982757 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb78cf-381a-43c2-9745-4c6e136dfb8a None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:09.983034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb78cf-381a-43c2-9745-4c6e136dfb8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:09.983838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb78cf-381a-43c2-9745-4c6e136dfb8a None admin] RBAC: Adding query filter params (interface=public, service_id=5d69eb5573ce40179f69851ae813bf9e, region_id=RegionOne) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:09.984153 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb78cf-381a-43c2-9745-4c6e136dfb8a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:09.985634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb78cf-381a-43c2-9745-4c6e136dfb8a None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:09.994611 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 388/763] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 24 07:33:09 2022] GET /identity/v3/endpoints?service_id=5d69eb5573ce40179f69851ae813bf9e&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:09.995608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb78cf-381a-43c2-9745-4c6e136dfb8a None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:10.012709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914074e-6667-4c21-a3c2-28a3b6c77652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:10.013324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914074e-6667-4c21-a3c2-28a3b6c77652 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:10.013597 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914074e-6667-4c21-a3c2-28a3b6c77652 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:10.013872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914074e-6667-4c21-a3c2-28a3b6c77652 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:10.086637 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914074e-6667-4c21-a3c2-28a3b6c77652 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:10.087267 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:10.088513 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914074e-6667-4c21-a3c2-28a3b6c77652 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:10.103434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb78cf-381a-43c2-9745-4c6e136dfb8a None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:10.138500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c7e24-9feb-47c5-901e-c8d742dfcbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:10.139367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c7e24-9feb-47c5-901e-c8d742dfcbfd None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:10.139650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c7e24-9feb-47c5-901e-c8d742dfcbfd None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:10.139928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c7e24-9feb-47c5-901e-c8d742dfcbfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:10.146438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c7e24-9feb-47c5-901e-c8d742dfcbfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:10.147888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c7e24-9feb-47c5-901e-c8d742dfcbfd None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:10.148024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-2b8c7e24-9feb-47c5-901e-c8d742dfcbfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:10.158942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:10.159884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:10.160796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:10.160796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:10.160796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:10.160796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:10.160796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:10.160796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:10.160796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:10.160796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:10.160796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 389/764] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:10.161758 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c7e24-9feb-47c5-901e-c8d742dfcbfd None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:10.192071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 376/765] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:33:10.193293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc482c3-5978-4bf6-a470-82b6bd3b5b7d None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:10.193573 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc482c3-5978-4bf6-a470-82b6bd3b5b7d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:10.193847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc482c3-5978-4bf6-a470-82b6bd3b5b7d None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:10.195224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 377/766] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:10.200081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0f120-c381-4888-9f79-df4636c2c54e None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:10.200364 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0f120-c381-4888-9f79-df4636c2c54e None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:10.200661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0f120-c381-4888-9f79-df4636c2c54e None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:10.201891 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 378/767] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:10.202589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0f120-c381-4888-9f79-df4636c2c54e None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:10.260305 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c7e24-9feb-47c5-901e-c8d742dfcbfd None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:10.292265 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2e7fd-77e7-4552-92ce-576afaf883fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:10.292894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2e7fd-77e7-4552-92ce-576afaf883fc None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:10.293176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2e7fd-77e7-4552-92ce-576afaf883fc None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:10.293448 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2e7fd-77e7-4552-92ce-576afaf883fc None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:10.294052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2e7fd-77e7-4552-92ce-576afaf883fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:10.295404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2e7fd-77e7-4552-92ce-576afaf883fc None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:10.300230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 390/768] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 24 07:33:10 2022] GET /identity/v3/services => generated 772 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:10.301653 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2e7fd-77e7-4552-92ce-576afaf883fc None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:10.312289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0f120-c381-4888-9f79-df4636c2c54e None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:10.324241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:10.352450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22294972-6e09-41e5-bc7d-5ae1b966aacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:10.353218 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22294972-6e09-41e5-bc7d-5ae1b966aacd None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:10.353506 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22294972-6e09-41e5-bc7d-5ae1b966aacd None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:10.353780 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22294972-6e09-41e5-bc7d-5ae1b966aacd None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:10.354507 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22294972-6e09-41e5-bc7d-5ae1b966aacd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:10.354900 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22294972-6e09-41e5-bc7d-5ae1b966aacd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:10.356396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22294972-6e09-41e5-bc7d-5ae1b966aacd None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:10.374541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 379/769] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:10.374541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e8556-41ce-439b-8a14-b6b75224a42d None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:10.374541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e8556-41ce-439b-8a14-b6b75224a42d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:10.374541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e8556-41ce-439b-8a14-b6b75224a42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:10.390751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-808e8556-41ce-439b-8a14-b6b75224a42d None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:10.391979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-808e8556-41ce-439b-8a14-b6b75224a42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:10.410729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:10.429809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2e7fd-77e7-4552-92ce-576afaf883fc None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:10.467332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40d12c-8dba-4a8f-90cf-d89735664b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:10.468209 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40d12c-8dba-4a8f-90cf-d89735664b8c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:10.468513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40d12c-8dba-4a8f-90cf-d89735664b8c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:10.468819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40d12c-8dba-4a8f-90cf-d89735664b8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:10.476162 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 380/770] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:10.477459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a6790-169e-41d3-a8b2-c0943b43b7c8 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:10.477748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a6790-169e-41d3-a8b2-c0943b43b7c8 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:10.478027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a6790-169e-41d3-a8b2-c0943b43b7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:10.480121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40d12c-8dba-4a8f-90cf-d89735664b8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:10.481603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40d12c-8dba-4a8f-90cf-d89735664b8c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-6e40d12c-8dba-4a8f-90cf-d89735664b8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:10.501175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:10.502166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:10.503036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:10.503036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:10.503036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:10.503036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 391/771] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:10.503036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40d12c-8dba-4a8f-90cf-d89735664b8c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:10.509445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-591a6790-169e-41d3-a8b2-c0943b43b7c8 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:10.510945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-591a6790-169e-41d3-a8b2-c0943b43b7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:10.576730 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:10.588698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 381/772] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:10.588698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a6790-169e-41d3-a8b2-c0943b43b7c8 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:10.764588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:10.789241 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a6790-169e-41d3-a8b2-c0943b43b7c8 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:10.822478 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40d12c-8dba-4a8f-90cf-d89735664b8c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:10.833535 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb045e-618a-4513-bf57-7506d1b41594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:10.834458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb045e-618a-4513-bf57-7506d1b41594 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:10.834742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb045e-618a-4513-bf57-7506d1b41594 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:10.835174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb045e-618a-4513-bf57-7506d1b41594 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:10.835951 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb045e-618a-4513-bf57-7506d1b41594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:10.836257 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb045e-618a-4513-bf57-7506d1b41594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:10.837942 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb045e-618a-4513-bf57-7506d1b41594 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:10.854340 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 382/773] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:10.856256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb045e-618a-4513-bf57-7506d1b41594 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:10.859727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf93ec-0ad0-4260-9593-0445f8b60771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:10.860684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf93ec-0ad0-4260-9593-0445f8b60771 None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:10.861185 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf93ec-0ad0-4260-9593-0445f8b60771 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:10.861656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf93ec-0ad0-4260-9593-0445f8b60771 None admin] PATH_INFO: `/v3/projects/fc060764023e44a7ade80eeed98a235f/users/bdf81f09006a42299c0807f77f16c748/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:10.864663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf93ec-0ad0-4260-9593-0445f8b60771 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc060764023e44a7ade80eeed98a235f, user_id=bdf81f09006a42299c0807f77f16c748, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:10.866305 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf93ec-0ad0-4260-9593-0445f8b60771 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:10.948674 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 392/774] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:33:10 2022] PUT /identity/v3/projects/fc060764023e44a7ade80eeed98a235f/users/bdf81f09006a42299c0807f77f16c748/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:33:10.948674 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf93ec-0ad0-4260-9593-0445f8b60771 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:10.973080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb045e-618a-4513-bf57-7506d1b41594 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:11.016299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f34208-3f79-43f8-b343-ca40ab7351ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:11.017102 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f34208-3f79-43f8-b343-ca40ab7351ff None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:11.017490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f34208-3f79-43f8-b343-ca40ab7351ff None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:11.017838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f34208-3f79-43f8-b343-ca40ab7351ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:11.030642 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f34208-3f79-43f8-b343-ca40ab7351ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-15f34208-3f79-43f8-b343-ca40ab7351ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:11.036300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:11.042777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:11.043667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:11.043667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:11.043667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:11.043667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:11.043667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:11.043667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:11.043667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:11.043667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:11.043667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:11.043667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:11.043667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:11.043667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:11.043667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 383/775] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:33:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:11.043667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f34208-3f79-43f8-b343-ca40ab7351ff None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:11.049431 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:11.096926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf93ec-0ad0-4260-9593-0445f8b60771 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:11.130855 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:11.149604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f0a82-b1aa-4b0c-8f6e-bc50f8f7e27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:11.150226 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f0a82-b1aa-4b0c-8f6e-bc50f8f7e27e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:11.150508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f0a82-b1aa-4b0c-8f6e-bc50f8f7e27e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:11.150778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f0a82-b1aa-4b0c-8f6e-bc50f8f7e27e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:11.161256 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f0a82-b1aa-4b0c-8f6e-bc50f8f7e27e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:11.162697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f0a82-b1aa-4b0c-8f6e-bc50f8f7e27e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:11.176073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f34208-3f79-43f8-b343-ca40ab7351ff None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-e68f0a82-b1aa-4b0c-8f6e-bc50f8f7e27e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:11.177047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:11.181281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:11.182178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:11.182178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:11.182178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:11.182178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:11.182178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:11.182178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:11.182178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:11.182178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:11.182178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 393/776] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:11.182178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f0a82-b1aa-4b0c-8f6e-bc50f8f7e27e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:11.225188 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180caf98-a7ff-49f4-92f8-33024ba426c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:11.226584 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180caf98-a7ff-49f4-92f8-33024ba426c3 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:11.227068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180caf98-a7ff-49f4-92f8-33024ba426c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:11.227811 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180caf98-a7ff-49f4-92f8-33024ba426c3 None admin] PATH_INFO: `/v3/services/network` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:11.228905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180caf98-a7ff-49f4-92f8-33024ba426c3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:11.231068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180caf98-a7ff-49f4-92f8-33024ba426c3 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-180caf98-a7ff-49f4-92f8-33024ba426c3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:11.243754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:11.244777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:11.245630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:11.245630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:11.245630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:11.245630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:11.245630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:11.245630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:11.245630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:11.245630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:11.245630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:11.245630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:11.245630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:11.245630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:11.245630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:33:11.245630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:11.246355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 384/777] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:11 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:11.247208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180caf98-a7ff-49f4-92f8-33024ba426c3 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:11.293454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f0a82-b1aa-4b0c-8f6e-bc50f8f7e27e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:11.329974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b19ac8-46ee-4df8-9cf7-aabfcbe24dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:11.330586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19ac8-46ee-4df8-9cf7-aabfcbe24dfe None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:11.330867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19ac8-46ee-4df8-9cf7-aabfcbe24dfe None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:11.331143 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19ac8-46ee-4df8-9cf7-aabfcbe24dfe None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:11.332037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b19ac8-46ee-4df8-9cf7-aabfcbe24dfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:11.332347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b19ac8-46ee-4df8-9cf7-aabfcbe24dfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:11.333761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b19ac8-46ee-4df8-9cf7-aabfcbe24dfe None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:11.337036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:11.343158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 394/778] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:33:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:11.344002 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b19ac8-46ee-4df8-9cf7-aabfcbe24dfe None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:11.382468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180caf98-a7ff-49f4-92f8-33024ba426c3 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:11.392409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:11.428911 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd989ab-e9a3-4d7d-80a0-e101ba32d713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:11.430117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd989ab-e9a3-4d7d-80a0-e101ba32d713 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:11.430759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd989ab-e9a3-4d7d-80a0-e101ba32d713 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:11.431257 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd989ab-e9a3-4d7d-80a0-e101ba32d713 None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:11.432396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd989ab-e9a3-4d7d-80a0-e101ba32d713 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:11.432946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd989ab-e9a3-4d7d-80a0-e101ba32d713 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:11.434990 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd989ab-e9a3-4d7d-80a0-e101ba32d713 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:11.450440 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b19ac8-46ee-4df8-9cf7-aabfcbe24dfe None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:11.451187 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 385/779] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:11.452062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd989ab-e9a3-4d7d-80a0-e101ba32d713 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:11.492498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8b1ce-b560-4a6a-841a-0a2f667ab2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:11.493739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8b1ce-b560-4a6a-841a-0a2f667ab2c2 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:11.494249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8b1ce-b560-4a6a-841a-0a2f667ab2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:11.494775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8b1ce-b560-4a6a-841a-0a2f667ab2c2 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:11.496705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8b1ce-b560-4a6a-841a-0a2f667ab2c2 None admin] RBAC: Adding query filter params (name=network) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:11.497069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8b1ce-b560-4a6a-841a-0a2f667ab2c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:11.498449 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8b1ce-b560-4a6a-841a-0a2f667ab2c2 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:11.506061 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 395/780] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 24 07:33:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:11.507542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8b1ce-b560-4a6a-841a-0a2f667ab2c2 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:11.569454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd989ab-e9a3-4d7d-80a0-e101ba32d713 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:11.605314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:11.614210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff154e3-ed6f-46d7-bb59-c7da65c046d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:11.615327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff154e3-ed6f-46d7-bb59-c7da65c046d0 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:11.615327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff154e3-ed6f-46d7-bb59-c7da65c046d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:11.615782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff154e3-ed6f-46d7-bb59-c7da65c046d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:11.626645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff154e3-ed6f-46d7-bb59-c7da65c046d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:11.628288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff154e3-ed6f-46d7-bb59-c7da65c046d0 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:11.637197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8b1ce-b560-4a6a-841a-0a2f667ab2c2 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-4ff154e3-ed6f-46d7-bb59-c7da65c046d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:11.643725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:11.645498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:11.646707 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:11.646707 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:11.646707 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:11.648338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 386/781] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:11.648854 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:11.649572 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff154e3-ed6f-46d7-bb59-c7da65c046d0 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:11.684049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b919d97-db2e-4342-a914-9a4695238d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:11.684711 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b919d97-db2e-4342-a914-9a4695238d5e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:11.684996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b919d97-db2e-4342-a914-9a4695238d5e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:11.685264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b919d97-db2e-4342-a914-9a4695238d5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:11.686236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b919d97-db2e-4342-a914-9a4695238d5e None admin] RBAC: Adding query filter params (role.id=3acdf08298c84325b53a0e286cc53b30, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=be379e1a6add42b69e9fdd676efca280) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:11.686549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b919d97-db2e-4342-a914-9a4695238d5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:11.688699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b919d97-db2e-4342-a914-9a4695238d5e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:11.696554 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 396/782] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:11 2022] GET /identity/v3/role_assignments?user.id=be379e1a6add42b69e9fdd676efca280&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=3acdf08298c84325b53a0e286cc53b30 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:11.697434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b919d97-db2e-4342-a914-9a4695238d5e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:11.784070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff154e3-ed6f-46d7-bb59-c7da65c046d0 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:11.820005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b919d97-db2e-4342-a914-9a4695238d5e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:11.837509 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad136aa1-f4d2-4be1-810c-550762ad1532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:11.838179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad136aa1-f4d2-4be1-810c-550762ad1532 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:11.838415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad136aa1-f4d2-4be1-810c-550762ad1532 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:11.838700 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad136aa1-f4d2-4be1-810c-550762ad1532 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:11.839320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad136aa1-f4d2-4be1-810c-550762ad1532 None admin] RBAC: Adding query filter params (type=network) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:11.839629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad136aa1-f4d2-4be1-810c-550762ad1532 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:11.841069 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad136aa1-f4d2-4be1-810c-550762ad1532 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:11.847136 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 387/783] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 24 07:33:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:11.852902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad136aa1-f4d2-4be1-810c-550762ad1532 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:11.869410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5acc81-f1e1-4d4b-8208-4d9aefe43c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:11.870692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5acc81-f1e1-4d4b-8208-4d9aefe43c0a None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:11.871124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5acc81-f1e1-4d4b-8208-4d9aefe43c0a None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:11.871575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5acc81-f1e1-4d4b-8208-4d9aefe43c0a None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:11.872393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5acc81-f1e1-4d4b-8208-4d9aefe43c0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:11.872877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5acc81-f1e1-4d4b-8208-4d9aefe43c0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:11.874495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5acc81-f1e1-4d4b-8208-4d9aefe43c0a None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:11.897937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 397/784] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:11.904232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5acc81-f1e1-4d4b-8208-4d9aefe43c0a None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:11.911316 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:11.943428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:11.996449 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad136aa1-f4d2-4be1-810c-550762ad1532 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:12.039199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc0e2a-068c-4aae-b676-eac4f0795ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:12.040090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc0e2a-068c-4aae-b676-eac4f0795ee6 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:12.040531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc0e2a-068c-4aae-b676-eac4f0795ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:12.040973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc0e2a-068c-4aae-b676-eac4f0795ee6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:12.041915 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc0e2a-068c-4aae-b676-eac4f0795ee6 None admin] RBAC: Adding query filter params (interface=public, service_id=0201008cf4be4cdeaee15aa5e6f48e29, region_id=RegionOne) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:12.042363 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc0e2a-068c-4aae-b676-eac4f0795ee6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:12.043858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc0e2a-068c-4aae-b676-eac4f0795ee6 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:12.054949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 388/785] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 24 07:33:11 2022] GET /identity/v3/endpoints?service_id=0201008cf4be4cdeaee15aa5e6f48e29&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:12.059684 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc0e2a-068c-4aae-b676-eac4f0795ee6 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:12.111511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5acc81-f1e1-4d4b-8208-4d9aefe43c0a None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:12.180976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc0e2a-068c-4aae-b676-eac4f0795ee6 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:12.192883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:12.202022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598d724-03f0-4816-8303-39baa537f090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:12.213067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598d724-03f0-4816-8303-39baa537f090 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:12.213067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598d724-03f0-4816-8303-39baa537f090 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:12.213067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598d724-03f0-4816-8303-39baa537f090 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:12.221705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910f607-b2fe-45b4-8e68-a67e4b4e0afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:12.222968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598d724-03f0-4816-8303-39baa537f090 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:12.223367 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910f607-b2fe-45b4-8e68-a67e4b4e0afe None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:12.223862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910f607-b2fe-45b4-8e68-a67e4b4e0afe None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:12.224298 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910f607-b2fe-45b4-8e68-a67e4b4e0afe None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:12.225025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598d724-03f0-4816-8303-39baa537f090 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:12.225293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910f607-b2fe-45b4-8e68-a67e4b4e0afe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:12.227003 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910f607-b2fe-45b4-8e68-a67e4b4e0afe None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:12.233339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 389/786] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 24 07:33:12 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-b598d724-03f0-4816-8303-39baa537f090 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:12.246367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:12.247586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:12.248845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:12.248845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:12.248845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:12.248845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:12.248845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:12.248845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:12.248845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:12.248845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:12.248845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:33:12.248845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:12.250943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 398/787] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:33:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:12.258240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910f607-b2fe-45b4-8e68-a67e4b4e0afe None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:12.335976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b264867-b835-487a-bdc8-dde56a002413 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:12.336508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b264867-b835-487a-bdc8-dde56a002413 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:12.337519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b264867-b835-487a-bdc8-dde56a002413 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:12.339378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 399/788] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:12.346040 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ee465-c308-46b5-99b3-54eab55c05de None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:12.346660 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ee465-c308-46b5-99b3-54eab55c05de None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:12.348275 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ee465-c308-46b5-99b3-54eab55c05de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:12.388810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-5c2ee465-c308-46b5-99b3-54eab55c05de None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:12.390039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5c2ee465-c308-46b5-99b3-54eab55c05de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:12.417256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910f607-b2fe-45b4-8e68-a67e4b4e0afe None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:12.460248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa904e-c209-4d1f-b699-a87b235186a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:12.461249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa904e-c209-4d1f-b699-a87b235186a4 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:12.461877 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa904e-c209-4d1f-b699-a87b235186a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:12.462378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa904e-c209-4d1f-b699-a87b235186a4 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:12.463367 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa904e-c209-4d1f-b699-a87b235186a4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:12.463890 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa904e-c209-4d1f-b699-a87b235186a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:12.465660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa904e-c209-4d1f-b699-a87b235186a4 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:12.480724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 400/789] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:12.491493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 390/790] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Aug 24 07:33:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:12.492846 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ee465-c308-46b5-99b3-54eab55c05de None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:12.521045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:12.593712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ee465-c308-46b5-99b3-54eab55c05de None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:12.628771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf41a8-b554-49e9-b87f-62d8b7c6abf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:12.629399 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf41a8-b554-49e9-b87f-62d8b7c6abf9 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:12.629674 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf41a8-b554-49e9-b87f-62d8b7c6abf9 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:12.629943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf41a8-b554-49e9-b87f-62d8b7c6abf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:12.640230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf41a8-b554-49e9-b87f-62d8b7c6abf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:12.641634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf41a8-b554-49e9-b87f-62d8b7c6abf9 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-dccf41a8-b554-49e9-b87f-62d8b7c6abf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:12.656613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:12.657595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:12.658488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:12.658488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:12.658488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:12.664107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 401/791] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:12.665511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa904e-c209-4d1f-b699-a87b235186a4 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:12.696888 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:12.786325 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa904e-c209-4d1f-b699-a87b235186a4 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:12.828154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c33b38-07e4-4ea2-8cca-542b6a80338b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:12.828787 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c33b38-07e4-4ea2-8cca-542b6a80338b None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:12.829067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c33b38-07e4-4ea2-8cca-542b6a80338b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:12.829340 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c33b38-07e4-4ea2-8cca-542b6a80338b None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:12.829983 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c33b38-07e4-4ea2-8cca-542b6a80338b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:12.830284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c33b38-07e4-4ea2-8cca-542b6a80338b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:12.831717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c33b38-07e4-4ea2-8cca-542b6a80338b None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:12.839478 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf41a8-b554-49e9-b87f-62d8b7c6abf9 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:12.865441 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 391/792] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:12.879288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c33b38-07e4-4ea2-8cca-542b6a80338b None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:12.970236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:13.006299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:13.016825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c33b38-07e4-4ea2-8cca-542b6a80338b None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:13.021814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf41a8-b554-49e9-b87f-62d8b7c6abf9 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:13.093348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0b678-b6ac-4fe2-bb57-ec814062b543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:13.093981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0b678-b6ac-4fe2-bb57-ec814062b543 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:13.094260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0b678-b6ac-4fe2-bb57-ec814062b543 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:13.094532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0b678-b6ac-4fe2-bb57-ec814062b543 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:13.095114 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0b678-b6ac-4fe2-bb57-ec814062b543 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:13.096468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0b678-b6ac-4fe2-bb57-ec814062b543 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-7ef0b678-b6ac-4fe2-bb57-ec814062b543 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:13.117094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:13.119454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:13.121481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 402/793] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Aug 24 07:33:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:13.128793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0b678-b6ac-4fe2-bb57-ec814062b543 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:13.137114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a10fd6-1200-4160-bfdd-6ad64f7e5bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:13.138167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a10fd6-1200-4160-bfdd-6ad64f7e5bd1 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:13.138467 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a10fd6-1200-4160-bfdd-6ad64f7e5bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:13.138745 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a10fd6-1200-4160-bfdd-6ad64f7e5bd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:13.150109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a10fd6-1200-4160-bfdd-6ad64f7e5bd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:13.152519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a10fd6-1200-4160-bfdd-6ad64f7e5bd1 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-58a10fd6-1200-4160-bfdd-6ad64f7e5bd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:13.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:13.174628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:13.176137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:13.176137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:13.176137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:13.176137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:13.176137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:13.176137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:13.176137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:13.176137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:13.195768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 392/794] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:13.199423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a10fd6-1200-4160-bfdd-6ad64f7e5bd1 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:13.304466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0b678-b6ac-4fe2-bb57-ec814062b543 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:13.337151 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a10fd6-1200-4160-bfdd-6ad64f7e5bd1 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:13.348335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8072c2-df51-42e6-949a-7bbaf8afaae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:13.348984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8072c2-df51-42e6-949a-7bbaf8afaae9 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:13.349259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8072c2-df51-42e6-949a-7bbaf8afaae9 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:13.349525 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8072c2-df51-42e6-949a-7bbaf8afaae9 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:13.350570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8072c2-df51-42e6-949a-7bbaf8afaae9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:13.350874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8072c2-df51-42e6-949a-7bbaf8afaae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:13.352532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8072c2-df51-42e6-949a-7bbaf8afaae9 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:13.355084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:13.364737 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 403/795] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Aug 24 07:33:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:13.367334 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cc34d-31a4-4f00-82df-d15cfcb02c95 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:13.367871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cc34d-31a4-4f00-82df-d15cfcb02c95 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:13.368972 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cc34d-31a4-4f00-82df-d15cfcb02c95 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:13.371404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 404/796] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:13.375624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cc34d-31a4-4f00-82df-d15cfcb02c95 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:13.385175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fddf6-6c45-46b2-b40a-76bae694a770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:13.386066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fddf6-6c45-46b2-b40a-76bae694a770 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:13.386594 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fddf6-6c45-46b2-b40a-76bae694a770 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:13.387070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fddf6-6c45-46b2-b40a-76bae694a770 None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:13.387988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fddf6-6c45-46b2-b40a-76bae694a770 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:13.388500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fddf6-6c45-46b2-b40a-76bae694a770 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:13.390228 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fddf6-6c45-46b2-b40a-76bae694a770 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:13.404721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 393/797] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:13.406284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6f7ba-3654-4957-8a46-9ee8270189bc None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:13.406797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6f7ba-3654-4957-8a46-9ee8270189bc None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:13.407291 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6f7ba-3654-4957-8a46-9ee8270189bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:13.426973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-f9d6f7ba-3654-4957-8a46-9ee8270189bc None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:13.428707 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f9d6f7ba-3654-4957-8a46-9ee8270189bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:13.432507 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:13.508779 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 394/798] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:13.510185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6f7ba-3654-4957-8a46-9ee8270189bc None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:13.510799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cc34d-31a4-4f00-82df-d15cfcb02c95 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:13.548667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97b084-6538-4dcc-90ca-becd01ffc312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:13.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97b084-6538-4dcc-90ca-becd01ffc312 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:13.549561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97b084-6538-4dcc-90ca-becd01ffc312 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:13.549829 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97b084-6538-4dcc-90ca-becd01ffc312 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:13.550828 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97b084-6538-4dcc-90ca-becd01ffc312 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:13.551130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97b084-6538-4dcc-90ca-becd01ffc312 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:13.552485 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97b084-6538-4dcc-90ca-becd01ffc312 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:13.563767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 405/799] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Aug 24 07:33:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:13.564919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97b084-6538-4dcc-90ca-becd01ffc312 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:13.637917 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:13.680203 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97b084-6538-4dcc-90ca-becd01ffc312 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:13.701038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6f7ba-3654-4957-8a46-9ee8270189bc None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:13.718653 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c5bb1-8fb0-4d44-93c9-af9f0ce78c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:13.719675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c5bb1-8fb0-4d44-93c9-af9f0ce78c7f None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:13.719961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c5bb1-8fb0-4d44-93c9-af9f0ce78c7f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:13.720233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c5bb1-8fb0-4d44-93c9-af9f0ce78c7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:13.720979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c5bb1-8fb0-4d44-93c9-af9f0ce78c7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:13.723018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c5bb1-8fb0-4d44-93c9-af9f0ce78c7f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:13.759701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08608148-4889-43d3-a8c8-4fa0a1530202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:13.760480 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08608148-4889-43d3-a8c8-4fa0a1530202 None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:13.760792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08608148-4889-43d3-a8c8-4fa0a1530202 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:13.761065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08608148-4889-43d3-a8c8-4fa0a1530202 None admin] PATH_INFO: `/v3/projects/fc060764023e44a7ade80eeed98a235f/users/b629ad3e6b9f4723b103f9845df2aaaf/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:13.762999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08608148-4889-43d3-a8c8-4fa0a1530202 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc060764023e44a7ade80eeed98a235f, user_id=b629ad3e6b9f4723b103f9845df2aaaf, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:13.764556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08608148-4889-43d3-a8c8-4fa0a1530202 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:13.852897 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 406/800] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:33:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:33:13.873019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:13.874168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62830f-53f6-4999-ad2f-2ffad23e70fb None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:13.874702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62830f-53f6-4999-ad2f-2ffad23e70fb None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:13.875332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62830f-53f6-4999-ad2f-2ffad23e70fb None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:13.877556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 407/801] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:13.877825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 395/802] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:33:13 2022] PUT /identity/v3/projects/fc060764023e44a7ade80eeed98a235f/users/b629ad3e6b9f4723b103f9845df2aaaf/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:33:13.880077 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62830f-53f6-4999-ad2f-2ffad23e70fb None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:13.886632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a6e92-60c4-460c-ab6e-f734315a4416 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:13.887074 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a6e92-60c4-460c-ab6e-f734315a4416 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:13.887430 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a6e92-60c4-460c-ab6e-f734315a4416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:13.908879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d01a6e92-60c4-460c-ab6e-f734315a4416 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:13.911262 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d01a6e92-60c4-460c-ab6e-f734315a4416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:14.008215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 396/803] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:14.057834 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62830f-53f6-4999-ad2f-2ffad23e70fb None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:14.084771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:14.100488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867f1bc-7c50-4dcc-97e2-6cb04597c9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:14.101177 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867f1bc-7c50-4dcc-97e2-6cb04597c9fe None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:14.101464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867f1bc-7c50-4dcc-97e2-6cb04597c9fe None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:14.101802 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867f1bc-7c50-4dcc-97e2-6cb04597c9fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:14.115628 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867f1bc-7c50-4dcc-97e2-6cb04597c9fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-1867f1bc-7c50-4dcc-97e2-6cb04597c9fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:14.120772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:14.121775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:14.122637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:14.122637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:14.122637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:14.122637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:14.122637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:14.122637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:14.122637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:14.122637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:14.122637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:14.122637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:14.122637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:14.122637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:14.125745 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 408/804] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:33:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:14.131908 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a6e92-60c4-460c-ab6e-f734315a4416 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:14.243111 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a6e92-60c4-460c-ab6e-f734315a4416 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:14.271619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:14.281874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966bd16-c3ec-4c8c-8334-3e4ca1065981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:14.282488 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966bd16-c3ec-4c8c-8334-3e4ca1065981 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:14.282769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966bd16-c3ec-4c8c-8334-3e4ca1065981 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:14.283044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966bd16-c3ec-4c8c-8334-3e4ca1065981 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:14.283850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8966bd16-c3ec-4c8c-8334-3e4ca1065981 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:14.284157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8966bd16-c3ec-4c8c-8334-3e4ca1065981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:14.285951 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8966bd16-c3ec-4c8c-8334-3e4ca1065981 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:14.296485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 397/805] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:33:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:14.302231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867f1bc-7c50-4dcc-97e2-6cb04597c9fe None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:14.310145 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966bd16-c3ec-4c8c-8334-3e4ca1065981 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:14.543919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867f1bc-7c50-4dcc-97e2-6cb04597c9fe None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:14.570868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966bd16-c3ec-4c8c-8334-3e4ca1065981 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:14.590464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0efeb0b-5c05-4b01-9eb4-0e9ea1f9cf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:14.590925 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efeb0b-5c05-4b01-9eb4-0e9ea1f9cf19 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:14.591204 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efeb0b-5c05-4b01-9eb4-0e9ea1f9cf19 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:14.591477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efeb0b-5c05-4b01-9eb4-0e9ea1f9cf19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:14.606454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0efeb0b-5c05-4b01-9eb4-0e9ea1f9cf19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:14.607967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0efeb0b-5c05-4b01-9eb4-0e9ea1f9cf19 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:14.628938 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c145a81-9458-4db7-a7d6-e0e84ec2754a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:14.629839 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c145a81-9458-4db7-a7d6-e0e84ec2754a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:14.630267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c145a81-9458-4db7-a7d6-e0e84ec2754a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:14.630688 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c145a81-9458-4db7-a7d6-e0e84ec2754a None admin] PATH_INFO: `/v3/roles/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-b0efeb0b-5c05-4b01-9eb4-0e9ea1f9cf19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:14.636547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:14.637684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:14.638777 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:14.638777 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:14.638777 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:14.646527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 409/806] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:14.652695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0efeb0b-5c05-4b01-9eb4-0e9ea1f9cf19 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:14.654513 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c145a81-9458-4db7-a7d6-e0e84ec2754a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-6c145a81-9458-4db7-a7d6-e0e84ec2754a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:14.661982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:14.663118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:14.664150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:14.664150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:14.664150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:14.664150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:14.664150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:14.664150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:14.664150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:14.664150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:14.664150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:14.664150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:14.664150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:33:14.664150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:14.673831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 398/807] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:33:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:14.678778 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c145a81-9458-4db7-a7d6-e0e84ec2754a None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:14.702097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:14.737482 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:14.790079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0efeb0b-5c05-4b01-9eb4-0e9ea1f9cf19 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:14.797755 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c145a81-9458-4db7-a7d6-e0e84ec2754a None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:14.835068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccde0a-4789-42b2-a69c-e3df750bb8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:14.835670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccde0a-4789-42b2-a69c-e3df750bb8d0 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:14.835948 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccde0a-4789-42b2-a69c-e3df750bb8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:14.836221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccde0a-4789-42b2-a69c-e3df750bb8d0 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:14.836757 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c77773-0b75-4690-92b1-9babdedc095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:14.836959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccde0a-4789-42b2-a69c-e3df750bb8d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:14.837224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccde0a-4789-42b2-a69c-e3df750bb8d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:14.837985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c77773-0b75-4690-92b1-9babdedc095c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:14.838470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c77773-0b75-4690-92b1-9babdedc095c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:14.838645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccde0a-4789-42b2-a69c-e3df750bb8d0 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:14.839130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c77773-0b75-4690-92b1-9babdedc095c None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:14.840252 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c77773-0b75-4690-92b1-9babdedc095c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:14.841227 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c77773-0b75-4690-92b1-9babdedc095c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:14.842780 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c77773-0b75-4690-92b1-9babdedc095c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:14.848250 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 399/808] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:33:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:14.849697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c51bc-4159-49e8-a2e5-e5257e90cc2c None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:14.849988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c51bc-4159-49e8-a2e5-e5257e90cc2c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:14.850265 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c51bc-4159-49e8-a2e5-e5257e90cc2c None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:14.855670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 400/809] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:14.855670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c51bc-4159-49e8-a2e5-e5257e90cc2c None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:14.865050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 410/810] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:14.867503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083f30b-439e-4308-8392-98c8f65453c2 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:14.868352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083f30b-439e-4308-8392-98c8f65453c2 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:14.868893 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083f30b-439e-4308-8392-98c8f65453c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:14.892986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0083f30b-439e-4308-8392-98c8f65453c2 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:14.895037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0083f30b-439e-4308-8392-98c8f65453c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:14.964126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c51bc-4159-49e8-a2e5-e5257e90cc2c None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:14.979623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:14.996952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 411/811] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:14.996952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083f30b-439e-4308-8392-98c8f65453c2 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:15.031419 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30472c9e-b87d-41c1-a0a5-2b1e72315095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:15.032097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30472c9e-b87d-41c1-a0a5-2b1e72315095 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:15.032381 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30472c9e-b87d-41c1-a0a5-2b1e72315095 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:15.032671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30472c9e-b87d-41c1-a0a5-2b1e72315095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:15.037159 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:15.050329 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30472c9e-b87d-41c1-a0a5-2b1e72315095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:15.051755 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30472c9e-b87d-41c1-a0a5-2b1e72315095 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-30472c9e-b87d-41c1-a0a5-2b1e72315095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:15.079682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:15.080717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:15.081921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:15.081921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:15.081921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:15.082093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 401/812] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:15.086917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30472c9e-b87d-41c1-a0a5-2b1e72315095 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:15.174986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083f30b-439e-4308-8392-98c8f65453c2 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:15.249856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba123f5-7e90-4a90-9ea0-0036594ecdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:15.250499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba123f5-7e90-4a90-9ea0-0036594ecdc9 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:15.250782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba123f5-7e90-4a90-9ea0-0036594ecdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:15.251052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba123f5-7e90-4a90-9ea0-0036594ecdc9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:15.257540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30472c9e-b87d-41c1-a0a5-2b1e72315095 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:15.265503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba123f5-7e90-4a90-9ea0-0036594ecdc9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:15.266957 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba123f5-7e90-4a90-9ea0-0036594ecdc9 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-8ba123f5-7e90-4a90-9ea0-0036594ecdc9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:15.289193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:15.290193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:15.291152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:15.291152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:15.291152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:15.291152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:15.291152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:15.291152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:15.291152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:15.291152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:15.291152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:33:15.291152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:15.291152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 412/813] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 24 07:33:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:15.302339 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba123f5-7e90-4a90-9ea0-0036594ecdc9 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:15.334022 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef367d97-a287-43f2-9ffb-54742b9f6599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:15.334639 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef367d97-a287-43f2-9ffb-54742b9f6599 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:15.334918 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef367d97-a287-43f2-9ffb-54742b9f6599 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:15.335191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef367d97-a287-43f2-9ffb-54742b9f6599 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:15.335833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef367d97-a287-43f2-9ffb-54742b9f6599 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:15.336133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef367d97-a287-43f2-9ffb-54742b9f6599 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:15.337737 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef367d97-a287-43f2-9ffb-54742b9f6599 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:15.358067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 402/814] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:15.364307 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef367d97-a287-43f2-9ffb-54742b9f6599 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:15.395781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:15.435600 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:15.457388 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba123f5-7e90-4a90-9ea0-0036594ecdc9 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:15.493350 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef367d97-a287-43f2-9ffb-54742b9f6599 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:15.500662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c122639-1e96-4324-9abd-7835d90c0ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:15.501822 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c122639-1e96-4324-9abd-7835d90c0ed0 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:15.502328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c122639-1e96-4324-9abd-7835d90c0ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:15.502747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c122639-1e96-4324-9abd-7835d90c0ed0 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:15.504214 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c122639-1e96-4324-9abd-7835d90c0ed0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:15.504708 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c122639-1e96-4324-9abd-7835d90c0ed0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:15.506964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c122639-1e96-4324-9abd-7835d90c0ed0 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:15.532525 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 413/815] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 24 07:33:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:15.533637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c122639-1e96-4324-9abd-7835d90c0ed0 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:15.537384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75def557-32e7-4343-bba6-b1525029b590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:15.538021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75def557-32e7-4343-bba6-b1525029b590 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:15.538302 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75def557-32e7-4343-bba6-b1525029b590 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:15.538574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75def557-32e7-4343-bba6-b1525029b590 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:15.548037 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75def557-32e7-4343-bba6-b1525029b590 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:15.549610 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75def557-32e7-4343-bba6-b1525029b590 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-75def557-32e7-4343-bba6-b1525029b590 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:15.563986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:15.565191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:15.566231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:15.566231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:15.566231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:15.566231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:15.566231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:15.566231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:15.566231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:15.566231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:15.566231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:33:15.566231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:15.568414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 403/816] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 24 07:33:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:15.569385 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75def557-32e7-4343-bba6-b1525029b590 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:15.665423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:15.667447 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c122639-1e96-4324-9abd-7835d90c0ed0 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:15.688937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75def557-32e7-4343-bba6-b1525029b590 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:15.711412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11031bc6-6899-41c5-969e-301338cd1370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:15.712070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11031bc6-6899-41c5-969e-301338cd1370 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:15.712352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11031bc6-6899-41c5-969e-301338cd1370 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:15.712622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11031bc6-6899-41c5-969e-301338cd1370 None admin] PATH_INFO: `/v3/services/network` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:15.713533 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11031bc6-6899-41c5-969e-301338cd1370 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:15.715135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11031bc6-6899-41c5-969e-301338cd1370 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-11031bc6-6899-41c5-969e-301338cd1370 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:15.726782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:15.727804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:15.728856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 414/817] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:15 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:15.730098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11031bc6-6899-41c5-969e-301338cd1370 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:15.730670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bdadc-ceaa-472c-aca7-6fdb72718c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:15.731503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bdadc-ceaa-472c-aca7-6fdb72718c60 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:15.731983 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bdadc-ceaa-472c-aca7-6fdb72718c60 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:15.732434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:15.732719 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bdadc-ceaa-472c-aca7-6fdb72718c60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:15.810117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158bdadc-ceaa-472c-aca7-6fdb72718c60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:15.812271 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158bdadc-ceaa-472c-aca7-6fdb72718c60 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:15.859890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11031bc6-6899-41c5-969e-301338cd1370 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:15.900562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:15.906405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e4a64-b0d8-4d26-b7e1-def3ec1707a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:15.907020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e4a64-b0d8-4d26-b7e1-def3ec1707a0 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:15.907301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e4a64-b0d8-4d26-b7e1-def3ec1707a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:15.907574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e4a64-b0d8-4d26-b7e1-def3ec1707a0 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:15.918230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e4a64-b0d8-4d26-b7e1-def3ec1707a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:15.918230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e4a64-b0d8-4d26-b7e1-def3ec1707a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:15.918230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e4a64-b0d8-4d26-b7e1-def3ec1707a0 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:15.928629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 404/818] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:33:15.929822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bdadc-ceaa-472c-aca7-6fdb72718c60 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:15.936060 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 415/819] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Aug 24 07:33:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:15.937591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e4a64-b0d8-4d26-b7e1-def3ec1707a0 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:16.007574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:16.042710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bdadc-ceaa-472c-aca7-6fdb72718c60 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:16.048400 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e4a64-b0d8-4d26-b7e1-def3ec1707a0 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:16.082664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b8e8e-f60f-40a1-87b4-820237d677d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:16.083729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b8e8e-f60f-40a1-87b4-820237d677d9 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.084209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b8e8e-f60f-40a1-87b4-820237d677d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.084683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b8e8e-f60f-40a1-87b4-820237d677d9 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.085724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b8e8e-f60f-40a1-87b4-820237d677d9 None admin] RBAC: Adding query filter params (name=network) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:16.086234 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b8e8e-f60f-40a1-87b4-820237d677d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:16.086608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa3774-84bd-4a37-b4e5-e843fda75f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:16.087312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa3774-84bd-4a37-b4e5-e843fda75f5f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.087680 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa3774-84bd-4a37-b4e5-e843fda75f5f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.088027 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa3774-84bd-4a37-b4e5-e843fda75f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.088623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b8e8e-f60f-40a1-87b4-820237d677d9 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:16.094181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 405/820] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 24 07:33:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:16.095350 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b8e8e-f60f-40a1-87b4-820237d677d9 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:16.100443 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa3774-84bd-4a37-b4e5-e843fda75f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:16.101292 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa3774-84bd-4a37-b4e5-e843fda75f5f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-61aa3774-84bd-4a37-b4e5-e843fda75f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:16.117790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:16.118743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:16.119626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:16.119626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:16.119626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:16.119626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 416/821] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:16.119626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa3774-84bd-4a37-b4e5-e843fda75f5f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:16.149221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:16.207229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b8e8e-f60f-40a1-87b4-820237d677d9 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:16.232521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa3774-84bd-4a37-b4e5-e843fda75f5f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:16.242065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01b2ad-7fe6-43f1-9070-4823421f56d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:16.242686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01b2ad-7fe6-43f1-9070-4823421f56d5 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.242971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01b2ad-7fe6-43f1-9070-4823421f56d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.243241 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01b2ad-7fe6-43f1-9070-4823421f56d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.246260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:16.253199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01b2ad-7fe6-43f1-9070-4823421f56d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:16.254844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01b2ad-7fe6-43f1-9070-4823421f56d5 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:16.265620 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09222f-2f05-4297-a3b0-d2d49c74c25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:16.266270 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09222f-2f05-4297-a3b0-d2d49c74c25b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.266554 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09222f-2f05-4297-a3b0-d2d49c74c25b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.266832 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09222f-2f05-4297-a3b0-d2d49c74c25b None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.267462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09222f-2f05-4297-a3b0-d2d49c74c25b None admin] RBAC: Adding query filter params (type=network) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:16.267773 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09222f-2f05-4297-a3b0-d2d49c74c25b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-6e01b2ad-7fe6-43f1-9070-4823421f56d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:16.269462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:16.271171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:16.272542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:16.272542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:16.272542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:16.272749 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09222f-2f05-4297-a3b0-d2d49c74c25b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:16.272821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 406/822] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:16.272821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01b2ad-7fe6-43f1-9070-4823421f56d5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:16.280595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 417/823] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 24 07:33:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:16.280595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880553a-592a-4abd-afff-3f50b1b32bc2 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.280595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880553a-592a-4abd-afff-3f50b1b32bc2 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.280595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880553a-592a-4abd-afff-3f50b1b32bc2 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.280595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 418/824] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:16.280595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e06d80-1f61-4c04-8d5a-4920fb454e73 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.280595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e06d80-1f61-4c04-8d5a-4920fb454e73 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.281161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e06d80-1f61-4c04-8d5a-4920fb454e73 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.286580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 419/825] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:16.286580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e06d80-1f61-4c04-8d5a-4920fb454e73 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:16.367019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:16.370752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01b2ad-7fe6-43f1-9070-4823421f56d5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:16.385380 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e06d80-1f61-4c04-8d5a-4920fb454e73 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:16.404272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b4e7e-a844-49b7-903e-4940cf620642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:16.405145 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b4e7e-a844-49b7-903e-4940cf620642 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.405472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b4e7e-a844-49b7-903e-4940cf620642 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.405968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b4e7e-a844-49b7-903e-4940cf620642 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.406642 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b4e7e-a844-49b7-903e-4940cf620642 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:16.407304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b4e7e-a844-49b7-903e-4940cf620642 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:16.408690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b4e7e-a844-49b7-903e-4940cf620642 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:16.419059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1bffdc-e449-42df-8f23-bd1400e5c2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:16.419654 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bffdc-e449-42df-8f23-bd1400e5c2ec None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.419935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bffdc-e449-42df-8f23-bd1400e5c2ec None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.420207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bffdc-e449-42df-8f23-bd1400e5c2ec None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.420496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 407/826] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:16.420873 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1bffdc-e449-42df-8f23-bd1400e5c2ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:16.421176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1bffdc-e449-42df-8f23-bd1400e5c2ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:16.421333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b4e7e-a844-49b7-903e-4940cf620642 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:16.422520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1bffdc-e449-42df-8f23-bd1400e5c2ec None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:16.434269 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 420/827] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:16.435432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5032e8e-388f-47a1-a1e3-56a898d13790 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.435718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5032e8e-388f-47a1-a1e3-56a898d13790 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.435991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5032e8e-388f-47a1-a1e3-56a898d13790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.453376 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a5032e8e-388f-47a1-a1e3-56a898d13790 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:16.454563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a5032e8e-388f-47a1-a1e3-56a898d13790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:16.484200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:16.520930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b4e7e-a844-49b7-903e-4940cf620642 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:16.524777 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 421/828] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:16.526253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5f52a-1b29-4419-8dd9-bee2e64a19b0 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.526778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5f52a-1b29-4419-8dd9-bee2e64a19b0 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.527062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5f52a-1b29-4419-8dd9-bee2e64a19b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.545949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-11a5f52a-1b29-4419-8dd9-bee2e64a19b0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:16.547135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-11a5f52a-1b29-4419-8dd9-bee2e64a19b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:16.559862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849ce52-7426-4879-a3c8-704951427bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:16.560453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849ce52-7426-4879-a3c8-704951427bab None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.560782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849ce52-7426-4879-a3c8-704951427bab None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.561042 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849ce52-7426-4879-a3c8-704951427bab None admin] PATH_INFO: `/v3/endpoints` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.561702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849ce52-7426-4879-a3c8-704951427bab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:16.563061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849ce52-7426-4879-a3c8-704951427bab None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:16.614740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 422/829] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:16.615578 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5f52a-1b29-4419-8dd9-bee2e64a19b0 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:16.659048 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 408/830] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:33:16 2022] POST /identity/v3/endpoints => generated 337 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:33:16.659048 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849ce52-7426-4879-a3c8-704951427bab None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:16.692982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:16.772617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:16.777736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5f52a-1b29-4419-8dd9-bee2e64a19b0 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:16.844770 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b3b0b-6afd-4485-b9ea-6aea452412c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:16.845454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b3b0b-6afd-4485-b9ea-6aea452412c5 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.845733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b3b0b-6afd-4485-b9ea-6aea452412c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.846009 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b3b0b-6afd-4485-b9ea-6aea452412c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.849306 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849ce52-7426-4879-a3c8-704951427bab None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:16.854570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b3b0b-6afd-4485-b9ea-6aea452412c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:16.856019 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b3b0b-6afd-4485-b9ea-6aea452412c5 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-bc1b3b0b-6afd-4485-b9ea-6aea452412c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:16.870021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:16.870953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:16.871865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:16.871865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:16.871865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:16.871865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:16.871865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:16.871865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:16.871865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:16.871865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:16.871865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 423/831] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:16.871865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b3b0b-6afd-4485-b9ea-6aea452412c5 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:16.895215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7dcb7-1902-4b91-9a5b-eac134d0ae9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:16.895846 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7dcb7-1902-4b91-9a5b-eac134d0ae9c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:16.896126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7dcb7-1902-4b91-9a5b-eac134d0ae9c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:16.896397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7dcb7-1902-4b91-9a5b-eac134d0ae9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:16.904256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7dcb7-1902-4b91-9a5b-eac134d0ae9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:16.905126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7dcb7-1902-4b91-9a5b-eac134d0ae9c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-7de7dcb7-1902-4b91-9a5b-eac134d0ae9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:16.916752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:16.917646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:16.918604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:16.918604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:16.918604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:16.918604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:16.918604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:16.918604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:16.918604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:16.918604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:16.918604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 409/832] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:16.918604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7dcb7-1902-4b91-9a5b-eac134d0ae9c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:16.988939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b3b0b-6afd-4485-b9ea-6aea452412c5 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:16.995276 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:17.023246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83296b3b-d4c0-439f-97d8-197e1a52765d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:17.023875 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83296b3b-d4c0-439f-97d8-197e1a52765d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:17.024160 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83296b3b-d4c0-439f-97d8-197e1a52765d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:17.024435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83296b3b-d4c0-439f-97d8-197e1a52765d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:17.026238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7dcb7-1902-4b91-9a5b-eac134d0ae9c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:17.034387 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83296b3b-d4c0-439f-97d8-197e1a52765d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-83296b3b-d4c0-439f-97d8-197e1a52765d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:17.039769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:17.045605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:17.047038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:17.047038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:17.047038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:17.047038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:17.047038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:17.047038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:17.047038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:17.047038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:17.047038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:17.047038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:17.047038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:17.047038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:17.047038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 424/833] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:33:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:17.047038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83296b3b-d4c0-439f-97d8-197e1a52765d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:17.058342 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:17.062280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10749af4-22b1-48a6-93b7-d82c26438b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:17.062886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10749af4-22b1-48a6-93b7-d82c26438b1c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:17.063169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10749af4-22b1-48a6-93b7-d82c26438b1c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:17.063443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10749af4-22b1-48a6-93b7-d82c26438b1c None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:17.064129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10749af4-22b1-48a6-93b7-d82c26438b1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:17.064437 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10749af4-22b1-48a6-93b7-d82c26438b1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:17.065994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10749af4-22b1-48a6-93b7-d82c26438b1c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:17.077719 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 410/834] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:17.078536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10749af4-22b1-48a6-93b7-d82c26438b1c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:17.141807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83296b3b-d4c0-439f-97d8-197e1a52765d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:17.174984 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10749af4-22b1-48a6-93b7-d82c26438b1c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:17.175459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4843c4-6d1e-4c7c-bfef-ebaf63e692df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:17.176266 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4843c4-6d1e-4c7c-bfef-ebaf63e692df None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:17.176546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4843c4-6d1e-4c7c-bfef-ebaf63e692df None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:17.176843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4843c4-6d1e-4c7c-bfef-ebaf63e692df None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:17.177416 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4843c4-6d1e-4c7c-bfef-ebaf63e692df None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:17.178765 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4843c4-6d1e-4c7c-bfef-ebaf63e692df None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-0d4843c4-6d1e-4c7c-bfef-ebaf63e692df None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:17.187261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:17.188329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:17.189189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 425/835] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:33:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:17.189970 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4843c4-6d1e-4c7c-bfef-ebaf63e692df None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:17.206358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:17.212448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491777d-8f3f-4200-9c33-373c82dc117d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:17.213142 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491777d-8f3f-4200-9c33-373c82dc117d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:17.213407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491777d-8f3f-4200-9c33-373c82dc117d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:17.213635 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491777d-8f3f-4200-9c33-373c82dc117d None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:17.214326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491777d-8f3f-4200-9c33-373c82dc117d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:17.214710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491777d-8f3f-4200-9c33-373c82dc117d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:17.216089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491777d-8f3f-4200-9c33-373c82dc117d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:17.227623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 411/836] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:17.228450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491777d-8f3f-4200-9c33-373c82dc117d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:17.271500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:17.291969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4843c4-6d1e-4c7c-bfef-ebaf63e692df None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:17.325422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491777d-8f3f-4200-9c33-373c82dc117d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:17.326089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c9658-e82c-48fb-bc27-81534a042ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:17.326691 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c9658-e82c-48fb-bc27-81534a042ea5 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:17.326975 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c9658-e82c-48fb-bc27-81534a042ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:17.327249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c9658-e82c-48fb-bc27-81534a042ea5 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:17.327909 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c9658-e82c-48fb-bc27-81534a042ea5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:17.328244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c9658-e82c-48fb-bc27-81534a042ea5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:17.329635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c9658-e82c-48fb-bc27-81534a042ea5 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:17.337722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 426/837] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:33:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:17.338497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c9658-e82c-48fb-bc27-81534a042ea5 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:17.364437 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ceb22f-2a55-4029-8459-8234ecae00e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:17.365068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceb22f-2a55-4029-8459-8234ecae00e6 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:17.365351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceb22f-2a55-4029-8459-8234ecae00e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:17.365628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceb22f-2a55-4029-8459-8234ecae00e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:17.366348 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ceb22f-2a55-4029-8459-8234ecae00e6 None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=bdf81f09006a42299c0807f77f16c748) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:17.366886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ceb22f-2a55-4029-8459-8234ecae00e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:17.368084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ceb22f-2a55-4029-8459-8234ecae00e6 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:17.373950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 412/838] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:17 2022] GET /identity/v3/role_assignments?user.id=bdf81f09006a42299c0807f77f16c748&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:17.374759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ceb22f-2a55-4029-8459-8234ecae00e6 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:17.439389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:17.443956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c9658-e82c-48fb-bc27-81534a042ea5 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:17.482642 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d95c4-d3ea-4369-a3a4-7286034ac14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:17.483265 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d95c4-d3ea-4369-a3a4-7286034ac14e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:17.483553 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d95c4-d3ea-4369-a3a4-7286034ac14e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:17.483842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d95c4-d3ea-4369-a3a4-7286034ac14e None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:17.484764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d95c4-d3ea-4369-a3a4-7286034ac14e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:17.484896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d95c4-d3ea-4369-a3a4-7286034ac14e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:17.486184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d95c4-d3ea-4369-a3a4-7286034ac14e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:17.496428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 427/839] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:33:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:17.496630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ceb22f-2a55-4029-8459-8234ecae00e6 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:17.496756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d95c4-d3ea-4369-a3a4-7286034ac14e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:17.499676 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:17.537993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b4129-64ef-4c1c-a9fc-2f7297be2e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:17.538185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b4129-64ef-4c1c-a9fc-2f7297be2e64 None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:17.538329 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b4129-64ef-4c1c-a9fc-2f7297be2e64 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:17.538601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b4129-64ef-4c1c-a9fc-2f7297be2e64 None admin] PATH_INFO: `/v3/projects/fc060764023e44a7ade80eeed98a235f/users/be379e1a6add42b69e9fdd676efca280/roles/3acdf08298c84325b53a0e286cc53b30` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:17.558185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b4129-64ef-4c1c-a9fc-2f7297be2e64 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc060764023e44a7ade80eeed98a235f, user_id=be379e1a6add42b69e9fdd676efca280, role_id=3acdf08298c84325b53a0e286cc53b30)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:17.559634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b4129-64ef-4c1c-a9fc-2f7297be2e64 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:17.609635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d95c4-d3ea-4369-a3a4-7286034ac14e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:17.635853 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 413/840] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:33:17 2022] PUT /identity/v3/projects/fc060764023e44a7ade80eeed98a235f/users/be379e1a6add42b69e9fdd676efca280/roles/3acdf08298c84325b53a0e286cc53b30 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:33:17.636710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b4129-64ef-4c1c-a9fc-2f7297be2e64 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:17.654360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb1225-c7c5-48c8-8e92-5f8988039d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:17.655578 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb1225-c7c5-48c8-8e92-5f8988039d86 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:17.656008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb1225-c7c5-48c8-8e92-5f8988039d86 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:17.656590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb1225-c7c5-48c8-8e92-5f8988039d86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:17.673020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb1225-c7c5-48c8-8e92-5f8988039d86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:17.675421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb1225-c7c5-48c8-8e92-5f8988039d86 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-d8cb1225-c7c5-48c8-8e92-5f8988039d86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:17.694192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:17.695353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:17.696383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:17.696383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:17.696383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:17.698339 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 428/841] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:17.703964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb1225-c7c5-48c8-8e92-5f8988039d86 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:17.763036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b4129-64ef-4c1c-a9fc-2f7297be2e64 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:17.784245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:17.794586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:17.820226 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a44052-679a-4cc5-9a90-798962acfca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:17.821337 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a44052-679a-4cc5-9a90-798962acfca9 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:17.821836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a44052-679a-4cc5-9a90-798962acfca9 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:17.822308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a44052-679a-4cc5-9a90-798962acfca9 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:17.823577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a44052-679a-4cc5-9a90-798962acfca9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:17.824338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a44052-679a-4cc5-9a90-798962acfca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:17.826444 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a44052-679a-4cc5-9a90-798962acfca9 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:17.833630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 414/842] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:33:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:17.836425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb1225-c7c5-48c8-8e92-5f8988039d86 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:17.880769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae18fa0-5cb8-4b4c-bb36-83c6579c9f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:17.881399 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae18fa0-5cb8-4b4c-bb36-83c6579c9f3d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:17.881677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae18fa0-5cb8-4b4c-bb36-83c6579c9f3d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:17.881949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae18fa0-5cb8-4b4c-bb36-83c6579c9f3d None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:17.888558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae18fa0-5cb8-4b4c-bb36-83c6579c9f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:17.888558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae18fa0-5cb8-4b4c-bb36-83c6579c9f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:17.888558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae18fa0-5cb8-4b4c-bb36-83c6579c9f3d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:17.902909 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 429/843] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:17.908673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a44052-679a-4cc5-9a90-798962acfca9 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:18.023503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a44052-679a-4cc5-9a90-798962acfca9 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:18.060519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c009c8-4dbc-4c4e-b7fc-a3ff355b4404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:18.061567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c009c8-4dbc-4c4e-b7fc-a3ff355b4404 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:18.062069 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c009c8-4dbc-4c4e-b7fc-a3ff355b4404 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:18.062566 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c009c8-4dbc-4c4e-b7fc-a3ff355b4404 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:18.071148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c009c8-4dbc-4c4e-b7fc-a3ff355b4404 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:18.072918 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c009c8-4dbc-4c4e-b7fc-a3ff355b4404 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-50c009c8-4dbc-4c4e-b7fc-a3ff355b4404 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:18.085492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:18.086452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:18.087420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:18.087420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:18.087420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:18.087420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:18.087420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:18.087420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:18.087420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:18.087420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:18.087420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:33:18.087420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:18.089960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 415/844] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:33:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:18.095842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae18fa0-5cb8-4b4c-bb36-83c6579c9f3d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:18.205210 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae18fa0-5cb8-4b4c-bb36-83c6579c9f3d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:18.222934 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:18.241505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc87c8-41f6-4565-b914-4791370b3677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:18.242116 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc87c8-41f6-4565-b914-4791370b3677 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:18.242398 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc87c8-41f6-4565-b914-4791370b3677 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:18.242723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc87c8-41f6-4565-b914-4791370b3677 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:18.243506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc87c8-41f6-4565-b914-4791370b3677 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:18.243865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc87c8-41f6-4565-b914-4791370b3677 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:18.245317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc87c8-41f6-4565-b914-4791370b3677 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:18.264376 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 430/845] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Aug 24 07:33:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:18.270513 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c009c8-4dbc-4c4e-b7fc-a3ff355b4404 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:18.301865 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:18.378985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c009c8-4dbc-4c4e-b7fc-a3ff355b4404 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:18.415581 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05c4fc-671b-4359-804d-bd4fe946adcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:18.416541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05c4fc-671b-4359-804d-bd4fe946adcc None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:18.416989 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05c4fc-671b-4359-804d-bd4fe946adcc None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:18.417415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05c4fc-671b-4359-804d-bd4fe946adcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:18.490536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05c4fc-671b-4359-804d-bd4fe946adcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:18.492276 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05c4fc-671b-4359-804d-bd4fe946adcc None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:18.520924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:18.599347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 416/846] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:18 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:33:18.605176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc87c8-41f6-4565-b914-4791370b3677 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:18.709222 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc87c8-41f6-4565-b914-4791370b3677 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:18.744847 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a5ba2-e87d-4eed-bcc4-2b82da0533db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:18.745472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a5ba2-e87d-4eed-bcc4-2b82da0533db None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:18.745749 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a5ba2-e87d-4eed-bcc4-2b82da0533db None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:18.746020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a5ba2-e87d-4eed-bcc4-2b82da0533db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:18.757784 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a5ba2-e87d-4eed-bcc4-2b82da0533db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:18.759228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a5ba2-e87d-4eed-bcc4-2b82da0533db None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-dd2a5ba2-e87d-4eed-bcc4-2b82da0533db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:18.774220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:18.775169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:18.776063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:18.776063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:18.776063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:18.776063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 431/847] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:18.782432 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05c4fc-671b-4359-804d-bd4fe946adcc None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:18.894057 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05c4fc-671b-4359-804d-bd4fe946adcc None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:18.921309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:18.929382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e1c2b-a8d8-4a57-a058-45a29c530479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:18.929995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e1c2b-a8d8-4a57-a058-45a29c530479 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:18.930278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e1c2b-a8d8-4a57-a058-45a29c530479 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:18.930556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e1c2b-a8d8-4a57-a058-45a29c530479 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:18.931205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e1c2b-a8d8-4a57-a058-45a29c530479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:18.931512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e1c2b-a8d8-4a57-a058-45a29c530479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:18.932941 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e1c2b-a8d8-4a57-a058-45a29c530479 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:18.947375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 417/848] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:18.953069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a5ba2-e87d-4eed-bcc4-2b82da0533db None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:18.982198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:19.057571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a5ba2-e87d-4eed-bcc4-2b82da0533db None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:19.093499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6e309-cd3c-4994-a8cf-218843d4acc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:19.094129 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6e309-cd3c-4994-a8cf-218843d4acc6 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:19.094410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6e309-cd3c-4994-a8cf-218843d4acc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:19.094684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6e309-cd3c-4994-a8cf-218843d4acc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:19.101960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6e309-cd3c-4994-a8cf-218843d4acc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:19.103404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6e309-cd3c-4994-a8cf-218843d4acc6 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-4ff6e309-cd3c-4994-a8cf-218843d4acc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:19.115052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:19.115949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:19.116894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:19.116894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:19.116894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:19.116894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:19.116894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:19.116894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:19.116894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:19.116894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:19.116894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 432/849] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:19.123756 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e1c2b-a8d8-4a57-a058-45a29c530479 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:19.230816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e1c2b-a8d8-4a57-a058-45a29c530479 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:19.266407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a958f2a7-776a-48f8-8344-7e290a9f1d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:19.267008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958f2a7-776a-48f8-8344-7e290a9f1d20 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:19.267291 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958f2a7-776a-48f8-8344-7e290a9f1d20 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:19.267568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958f2a7-776a-48f8-8344-7e290a9f1d20 None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:19.268258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a958f2a7-776a-48f8-8344-7e290a9f1d20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:19.268564 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a958f2a7-776a-48f8-8344-7e290a9f1d20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:19.270537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a958f2a7-776a-48f8-8344-7e290a9f1d20 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:19.282751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 418/850] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:19.289412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6e309-cd3c-4994-a8cf-218843d4acc6 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:19.378517 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:19.393649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6e309-cd3c-4994-a8cf-218843d4acc6 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:19.430157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562f7bb-d69e-46e5-81c7-1e0a2acc41d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:19.430771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562f7bb-d69e-46e5-81c7-1e0a2acc41d7 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:19.431055 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562f7bb-d69e-46e5-81c7-1e0a2acc41d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:19.431330 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562f7bb-d69e-46e5-81c7-1e0a2acc41d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:19.432133 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562f7bb-d69e-46e5-81c7-1e0a2acc41d7 None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=b629ad3e6b9f4723b103f9845df2aaaf) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:19.432452 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562f7bb-d69e-46e5-81c7-1e0a2acc41d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:19.433906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562f7bb-d69e-46e5-81c7-1e0a2acc41d7 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:19.443807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 433/851] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:19 2022] GET /identity/v3/role_assignments?user.id=b629ad3e6b9f4723b103f9845df2aaaf&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:19.473547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cd924-1105-45ab-8b98-9f8a6d5208ba None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:19.474069 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cd924-1105-45ab-8b98-9f8a6d5208ba None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:19.474315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cd924-1105-45ab-8b98-9f8a6d5208ba None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:19.476024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 419/852] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:19.481895 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bea5d-7523-4594-be96-e89d44eb21a4 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:19.482175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bea5d-7523-4594-be96-e89d44eb21a4 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:19.482367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bea5d-7523-4594-be96-e89d44eb21a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:19.502071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-017bea5d-7523-4594-be96-e89d44eb21a4 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:19.503897 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-017bea5d-7523-4594-be96-e89d44eb21a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:19.579461 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 434/853] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:19.748355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331df6a-c8b8-4e67-9e5e-860dcc4078cc None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:19.748780 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331df6a-c8b8-4e67-9e5e-860dcc4078cc None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:19.749164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331df6a-c8b8-4e67-9e5e-860dcc4078cc None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:19.750991 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 420/854] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:19.756985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1e5f5-39b1-4acd-b882-6a2af13dd288 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:19.757442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1e5f5-39b1-4acd-b882-6a2af13dd288 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:19.757729 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1e5f5-39b1-4acd-b882-6a2af13dd288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:19.776607 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-c4b1e5f5-39b1-4acd-b882-6a2af13dd288 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:19.777821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c4b1e5f5-39b1-4acd-b882-6a2af13dd288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:19.850155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ce45e-35ed-4edd-b573-5a7b1bfd8e87 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:19.850804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ce45e-35ed-4edd-b573-5a7b1bfd8e87 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:19.851282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ce45e-35ed-4edd-b573-5a7b1bfd8e87 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:19.852836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 435/855] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:19.853230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 421/856] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:19.857905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1e5f5-39b1-4acd-b882-6a2af13dd288 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:19.858405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41983475-0dae-49b0-aaeb-9c7c67f4e1f8 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:19.858614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41983475-0dae-49b0-aaeb-9c7c67f4e1f8 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:19.858895 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41983475-0dae-49b0-aaeb-9c7c67f4e1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:19.870450 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:19.880122 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-41983475-0dae-49b0-aaeb-9c7c67f4e1f8 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:19.881418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-41983475-0dae-49b0-aaeb-9c7c67f4e1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:19.941612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:19.960420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1e5f5-39b1-4acd-b882-6a2af13dd288 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:19.960962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 422/857] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:19.995787 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8adb89-6fa5-44f6-94ae-ce93244e1100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:19.996422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8adb89-6fa5-44f6-94ae-ce93244e1100 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:19.996729 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8adb89-6fa5-44f6-94ae-ce93244e1100 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:19.997007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8adb89-6fa5-44f6-94ae-ce93244e1100 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:19.997590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8adb89-6fa5-44f6-94ae-ce93244e1100 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:19.999037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8adb89-6fa5-44f6-94ae-ce93244e1100 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-ed8adb89-6fa5-44f6-94ae-ce93244e1100 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:20.008510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:20.009466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:20.010373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 436/858] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Aug 24 07:33:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:20.017602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41983475-0dae-49b0-aaeb-9c7c67f4e1f8 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:20.119281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41983475-0dae-49b0-aaeb-9c7c67f4e1f8 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:20.133670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8adb89-6fa5-44f6-94ae-ce93244e1100 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:20.155849 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa100e-8ce4-4d5d-b2d0-95d74a2a792b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:20.156912 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa100e-8ce4-4d5d-b2d0-95d74a2a792b None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:20.157414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa100e-8ce4-4d5d-b2d0-95d74a2a792b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:20.157902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa100e-8ce4-4d5d-b2d0-95d74a2a792b None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:20.158861 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa100e-8ce4-4d5d-b2d0-95d74a2a792b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:20.160086 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa100e-8ce4-4d5d-b2d0-95d74a2a792b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:20.162113 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa100e-8ce4-4d5d-b2d0-95d74a2a792b None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:20.167655 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 423/859] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 24 07:33:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:20.173403 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa100e-8ce4-4d5d-b2d0-95d74a2a792b None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:20.213387 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:20.227767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:20.238975 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8adb89-6fa5-44f6-94ae-ce93244e1100 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:20.276562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84d18d-f1ce-4c06-b87b-37b3e898e96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:20.277156 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84d18d-f1ce-4c06-b87b-37b3e898e96f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:20.277437 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84d18d-f1ce-4c06-b87b-37b3e898e96f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:20.277715 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84d18d-f1ce-4c06-b87b-37b3e898e96f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:20.288863 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84d18d-f1ce-4c06-b87b-37b3e898e96f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-2f84d18d-f1ce-4c06-b87b-37b3e898e96f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:20.293941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:20.295045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:20.295888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:20.295888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:20.295888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:20.295888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:20.295888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:20.295888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:20.295888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:20.295888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:20.295888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:20.295888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:20.295888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:33:20.295888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:20.295888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 437/860] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:33:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:20.295888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84d18d-f1ce-4c06-b87b-37b3e898e96f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:20.302599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa100e-8ce4-4d5d-b2d0-95d74a2a792b None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:20.337168 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b13aab-c550-4ddd-9bc2-537f232a20ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:20.337778 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b13aab-c550-4ddd-9bc2-537f232a20ce None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:20.338060 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b13aab-c550-4ddd-9bc2-537f232a20ce None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:20.338337 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b13aab-c550-4ddd-9bc2-537f232a20ce None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:20.338963 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b13aab-c550-4ddd-9bc2-537f232a20ce None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:20.339273 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b13aab-c550-4ddd-9bc2-537f232a20ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:20.340671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b13aab-c550-4ddd-9bc2-537f232a20ce None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:20.352682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 424/861] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 24 07:33:20 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:20.352682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b13aab-c550-4ddd-9bc2-537f232a20ce None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:20.425448 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84d18d-f1ce-4c06-b87b-37b3e898e96f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:20.473063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c3424-2a31-404e-b1ff-bd18bed2dd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:20.473799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c3424-2a31-404e-b1ff-bd18bed2dd6e None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:20.474151 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c3424-2a31-404e-b1ff-bd18bed2dd6e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:20.474500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c3424-2a31-404e-b1ff-bd18bed2dd6e None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:20.475301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c3424-2a31-404e-b1ff-bd18bed2dd6e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:20.476833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c3424-2a31-404e-b1ff-bd18bed2dd6e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:20.476975 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b13aab-c550-4ddd-9bc2-537f232a20ce None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:20.480784 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:20.488783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:20.517850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87e018-4347-48cc-9df2-2f6a42485ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:20.518463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87e018-4347-48cc-9df2-2f6a42485ba7 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:20.518747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87e018-4347-48cc-9df2-2f6a42485ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:20.519672 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87e018-4347-48cc-9df2-2f6a42485ba7 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:20.520412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87e018-4347-48cc-9df2-2f6a42485ba7 None admin] RBAC: Adding query filter params (name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:20.520886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87e018-4347-48cc-9df2-2f6a42485ba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:20.522256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87e018-4347-48cc-9df2-2f6a42485ba7 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:20.532528 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 425/862] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:33:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:20.539463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87e018-4347-48cc-9df2-2f6a42485ba7 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:20.544621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 438/863] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 24 07:33:20 2022] POST /identity/v3/services => generated 251 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:33:20.667940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87e018-4347-48cc-9df2-2f6a42485ba7 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:20.713777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de384b38-4272-4a5a-985e-b360ed2ac8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:20.714416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de384b38-4272-4a5a-985e-b360ed2ac8f5 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:20.714691 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de384b38-4272-4a5a-985e-b360ed2ac8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:20.714958 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de384b38-4272-4a5a-985e-b360ed2ac8f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:20.727825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de384b38-4272-4a5a-985e-b360ed2ac8f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:20.729312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de384b38-4272-4a5a-985e-b360ed2ac8f5 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-de384b38-4272-4a5a-985e-b360ed2ac8f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:20.744200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:20.745194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:20.746154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:20.746154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:20.746154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:20.746154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 426/864] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:20.752584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c3424-2a31-404e-b1ff-bd18bed2dd6e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:20.896671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c3424-2a31-404e-b1ff-bd18bed2dd6e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:20.927674 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a5776-5584-4c23-b55e-586e3eb50dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:20.928296 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a5776-5584-4c23-b55e-586e3eb50dfa None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:20.928586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a5776-5584-4c23-b55e-586e3eb50dfa None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:20.928903 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a5776-5584-4c23-b55e-586e3eb50dfa None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:20.929555 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a5776-5584-4c23-b55e-586e3eb50dfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:20.929867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a5776-5584-4c23-b55e-586e3eb50dfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:20.931288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a5776-5584-4c23-b55e-586e3eb50dfa None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:20.944162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 439/865] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:20.953750 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de384b38-4272-4a5a-985e-b360ed2ac8f5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:20.989113 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:21.077682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de384b38-4272-4a5a-985e-b360ed2ac8f5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:21.117497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a16e8c-b099-4dcc-9ef7-4f86abd1a7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:21.118138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a16e8c-b099-4dcc-9ef7-4f86abd1a7f8 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:21.118420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a16e8c-b099-4dcc-9ef7-4f86abd1a7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:21.118696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a16e8c-b099-4dcc-9ef7-4f86abd1a7f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:21.126249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a16e8c-b099-4dcc-9ef7-4f86abd1a7f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:21.127704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a16e8c-b099-4dcc-9ef7-4f86abd1a7f8 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-53a16e8c-b099-4dcc-9ef7-4f86abd1a7f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:21.140039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:21.141044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:21.141977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:21.141977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:21.141977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:21.141977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:21.141977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:21.141977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:21.141977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:21.141977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:21.141977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:33:21.141977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:21.141977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 427/866] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 24 07:33:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:21.148749 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a5776-5584-4c23-b55e-586e3eb50dfa None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:21.243427 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:21.320788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a5776-5584-4c23-b55e-586e3eb50dfa None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:21.370269 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537c1af-ce53-49bf-9d54-de549307902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:21.370889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537c1af-ce53-49bf-9d54-de549307902a None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:21.371175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537c1af-ce53-49bf-9d54-de549307902a None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:21.371454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537c1af-ce53-49bf-9d54-de549307902a None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:21.372194 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537c1af-ce53-49bf-9d54-de549307902a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:21.372506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537c1af-ce53-49bf-9d54-de549307902a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:21.374247 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537c1af-ce53-49bf-9d54-de549307902a None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:21.402658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 440/867] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Aug 24 07:33:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:21.414064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a16e8c-b099-4dcc-9ef7-4f86abd1a7f8 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:21.523347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:21.540344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eab715-33ae-468a-a2b7-8b83c2a74509 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:21.540627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eab715-33ae-468a-a2b7-8b83c2a74509 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:21.540950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eab715-33ae-468a-a2b7-8b83c2a74509 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:21.542399 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 441/868] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:21.548194 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116ae86-fa1f-4e5b-b59e-fbb181ba6912 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:21.548484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116ae86-fa1f-4e5b-b59e-fbb181ba6912 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:21.548858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116ae86-fa1f-4e5b-b59e-fbb181ba6912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:21.554860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a16e8c-b099-4dcc-9ef7-4f86abd1a7f8 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:21.577817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a116ae86-fa1f-4e5b-b59e-fbb181ba6912 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:21.579481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a116ae86-fa1f-4e5b-b59e-fbb181ba6912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:21.602966 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c07111-56ff-40ec-887a-fecc99f60981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:21.603892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c07111-56ff-40ec-887a-fecc99f60981 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:21.604006 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c07111-56ff-40ec-887a-fecc99f60981 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:21.604253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c07111-56ff-40ec-887a-fecc99f60981 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:21.673051 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 442/869] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:21 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:21.693738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c07111-56ff-40ec-887a-fecc99f60981 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:21.695212 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c07111-56ff-40ec-887a-fecc99f60981 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:21.775423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:21.801181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 428/870] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:21 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:33:21.806740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116ae86-fa1f-4e5b-b59e-fbb181ba6912 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:21.853437 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:21.919054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116ae86-fa1f-4e5b-b59e-fbb181ba6912 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:21.938443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c07111-56ff-40ec-887a-fecc99f60981 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:21.959904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e589de9-183d-47b4-b8c4-1b8fc7b26da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:21.960542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e589de9-183d-47b4-b8c4-1b8fc7b26da3 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:21.961426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e589de9-183d-47b4-b8c4-1b8fc7b26da3 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:21.961714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e589de9-183d-47b4-b8c4-1b8fc7b26da3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:21.977514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e589de9-183d-47b4-b8c4-1b8fc7b26da3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:21.978977 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e589de9-183d-47b4-b8c4-1b8fc7b26da3 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-9e589de9-183d-47b4-b8c4-1b8fc7b26da3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:22.000378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:22.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:22.002318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:22.002318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:22.002318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:22.002499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 443/871] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:22.008354 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e589de9-183d-47b4-b8c4-1b8fc7b26da3 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:22.050595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c07111-56ff-40ec-887a-fecc99f60981 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:22.089339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d399ed-e84c-423b-b7bf-9ebdcb280436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:22.090127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d399ed-e84c-423b-b7bf-9ebdcb280436 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:22.090411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d399ed-e84c-423b-b7bf-9ebdcb280436 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:22.091004 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d399ed-e84c-423b-b7bf-9ebdcb280436 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:22.091868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d399ed-e84c-423b-b7bf-9ebdcb280436 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:22.093454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d399ed-e84c-423b-b7bf-9ebdcb280436 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-52d399ed-e84c-423b-b7bf-9ebdcb280436 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:22.105922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:22.107955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:22.109759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:22.109759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:22.109759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:22.109759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:22.109759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:22.109759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:22.109759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:22.109759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:22.109759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:22.109759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:22.109759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:22.109759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:22.109759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:22.109759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:22.112463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 429/872] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 24 07:33:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:22.114466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d399ed-e84c-423b-b7bf-9ebdcb280436 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:22.137398 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:22.144556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e589de9-183d-47b4-b8c4-1b8fc7b26da3 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:22.163860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:22.182978 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f7663-b40f-4338-816b-76db10296282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:22.184051 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f7663-b40f-4338-816b-76db10296282 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:22.184830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f7663-b40f-4338-816b-76db10296282 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:22.185535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f7663-b40f-4338-816b-76db10296282 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:22.186390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f7663-b40f-4338-816b-76db10296282 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:22.187019 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f7663-b40f-4338-816b-76db10296282 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:22.189232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f7663-b40f-4338-816b-76db10296282 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:22.210777 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 444/873] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:22.212251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f7663-b40f-4338-816b-76db10296282 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:22.237226 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d399ed-e84c-423b-b7bf-9ebdcb280436 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:22.274719 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6f3be-e115-4cc4-9212-4f489295830c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:22.275436 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6f3be-e115-4cc4-9212-4f489295830c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:22.275787 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6f3be-e115-4cc4-9212-4f489295830c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:22.276140 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6f3be-e115-4cc4-9212-4f489295830c None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:22.276902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6f3be-e115-4cc4-9212-4f489295830c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:22.277278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6f3be-e115-4cc4-9212-4f489295830c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:22.278729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6f3be-e115-4cc4-9212-4f489295830c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:22.288344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 430/874] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 24 07:33:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:22.290364 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6f3be-e115-4cc4-9212-4f489295830c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:22.343283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f7663-b40f-4338-816b-76db10296282 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:22.382683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38d853-c58e-441f-ab08-bbcf063d147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:22.383465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38d853-c58e-441f-ab08-bbcf063d147c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:22.383853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38d853-c58e-441f-ab08-bbcf063d147c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:22.384257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38d853-c58e-441f-ab08-bbcf063d147c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:22.391944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38d853-c58e-441f-ab08-bbcf063d147c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:22.393548 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38d853-c58e-441f-ab08-bbcf063d147c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-fc38d853-c58e-441f-ab08-bbcf063d147c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:22.407301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:22.408405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:22.409576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:22.409576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:22.409576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:22.409576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:22.409576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:22.409576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:22.409576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:22.409576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:22.416275 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 445/875] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:22.418132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38d853-c58e-441f-ab08-bbcf063d147c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:22.429782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6f3be-e115-4cc4-9212-4f489295830c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:22.432238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:22.438879 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:22.467743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488f1f7-6ca3-4d50-bc79-8de4f05d7f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:22.468274 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488f1f7-6ca3-4d50-bc79-8de4f05d7f65 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:22.468557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488f1f7-6ca3-4d50-bc79-8de4f05d7f65 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:22.468869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488f1f7-6ca3-4d50-bc79-8de4f05d7f65 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:22.469537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488f1f7-6ca3-4d50-bc79-8de4f05d7f65 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:22.469849 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488f1f7-6ca3-4d50-bc79-8de4f05d7f65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:22.471295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488f1f7-6ca3-4d50-bc79-8de4f05d7f65 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:22.482484 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 431/876] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 24 07:33:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:22.502421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f842d-4ca6-4511-b198-b41c52739cee None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:22.502746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f842d-4ca6-4511-b198-b41c52739cee None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:22.503024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f842d-4ca6-4511-b198-b41c52739cee None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:22.505534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 432/877] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:22.511211 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cae59-cc5a-4446-9970-de9f3b02208d None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:22.511499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cae59-cc5a-4446-9970-de9f3b02208d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:22.511776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cae59-cc5a-4446-9970-de9f3b02208d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:22.532261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-b01cae59-cc5a-4446-9970-de9f3b02208d None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:22.533743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b01cae59-cc5a-4446-9970-de9f3b02208d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:22.558737 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38d853-c58e-441f-ab08-bbcf063d147c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:22.600419 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323e1a-98a2-4b08-bb1d-093a75e11477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:22.601421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323e1a-98a2-4b08-bb1d-093a75e11477 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:22.601918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323e1a-98a2-4b08-bb1d-093a75e11477 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:22.602384 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323e1a-98a2-4b08-bb1d-093a75e11477 None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:22.603352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8323e1a-98a2-4b08-bb1d-093a75e11477 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:22.603848 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8323e1a-98a2-4b08-bb1d-093a75e11477 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:22.605704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8323e1a-98a2-4b08-bb1d-093a75e11477 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:22.619882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 446/878] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:22.627299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 433/879] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:22 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:22.628954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323e1a-98a2-4b08-bb1d-093a75e11477 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:22.711169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:22.727009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875041c0-9225-4143-af8c-5cdc818e59a6 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:22.727009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875041c0-9225-4143-af8c-5cdc818e59a6 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:22.727009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875041c0-9225-4143-af8c-5cdc818e59a6 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:22.727009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 434/880] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:22.727009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e641-ee58-4c6a-955e-7a9aeba879e3 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:22.727009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e641-ee58-4c6a-955e-7a9aeba879e3 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:22.727577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e641-ee58-4c6a-955e-7a9aeba879e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:22.740305 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323e1a-98a2-4b08-bb1d-093a75e11477 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:22.746390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-6794e641-ee58-4c6a-955e-7a9aeba879e3 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:22.747613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6794e641-ee58-4c6a-955e-7a9aeba879e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:22.789039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319a796-af52-4ccf-8b3c-cc49a1bf8cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:22.789602 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319a796-af52-4ccf-8b3c-cc49a1bf8cbb None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:22.789882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319a796-af52-4ccf-8b3c-cc49a1bf8cbb None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:22.790161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319a796-af52-4ccf-8b3c-cc49a1bf8cbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:22.790881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319a796-af52-4ccf-8b3c-cc49a1bf8cbb None admin] RBAC: Adding query filter params (role.id=3acdf08298c84325b53a0e286cc53b30, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=be379e1a6add42b69e9fdd676efca280) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:22.791189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319a796-af52-4ccf-8b3c-cc49a1bf8cbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:22.792614 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319a796-af52-4ccf-8b3c-cc49a1bf8cbb None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:22.803189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 447/881] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:22 2022] GET /identity/v3/role_assignments?user.id=be379e1a6add42b69e9fdd676efca280&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=3acdf08298c84325b53a0e286cc53b30 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:22.827716 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 435/882] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:22 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:22.897008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319a796-af52-4ccf-8b3c-cc49a1bf8cbb None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:23.013787 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319a796-af52-4ccf-8b3c-cc49a1bf8cbb None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:23.051296 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e74ad-2eaf-499a-9b93-b4800340794f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:23.052182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e74ad-2eaf-499a-9b93-b4800340794f None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:23.052619 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e74ad-2eaf-499a-9b93-b4800340794f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:23.053095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e74ad-2eaf-499a-9b93-b4800340794f None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:23.053897 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e74ad-2eaf-499a-9b93-b4800340794f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:23.055503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e74ad-2eaf-499a-9b93-b4800340794f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:23.104242 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e641-ee58-4c6a-955e-7a9aeba879e3 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:23.132540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:23.134059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:23.139792 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 448/883] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 24 07:33:22 2022] POST /identity/v3/services => generated 269 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:33:23.225504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e641-ee58-4c6a-955e-7a9aeba879e3 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:23.276025 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276de098-bc97-4a5e-94a7-8ba278d39525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:23.277034 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276de098-bc97-4a5e-94a7-8ba278d39525 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:23.277545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276de098-bc97-4a5e-94a7-8ba278d39525 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:23.278049 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276de098-bc97-4a5e-94a7-8ba278d39525 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:23.278849 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276de098-bc97-4a5e-94a7-8ba278d39525 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:23.280464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276de098-bc97-4a5e-94a7-8ba278d39525 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-276de098-bc97-4a5e-94a7-8ba278d39525 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:23.290164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:23.291263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:23.292117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:23.292117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:23.292117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:23.292117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:23.292117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:23.292117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:23.292117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:23.292117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:23.292117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:23.292117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:23.292117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:23.292117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:23.292117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:33:23.292117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:23.294980 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 436/884] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:33:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:23.301010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e74ad-2eaf-499a-9b93-b4800340794f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:23.410343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e74ad-2eaf-499a-9b93-b4800340794f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:23.452021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f8325-a711-4f93-9e7d-aa399f473465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:23.452638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f8325-a711-4f93-9e7d-aa399f473465 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:23.452942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f8325-a711-4f93-9e7d-aa399f473465 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:23.453213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f8325-a711-4f93-9e7d-aa399f473465 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:23.453840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f8325-a711-4f93-9e7d-aa399f473465 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:23.454143 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f8325-a711-4f93-9e7d-aa399f473465 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:23.455506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f8325-a711-4f93-9e7d-aa399f473465 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:23.461600 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 449/885] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:33:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:23.466882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276de098-bc97-4a5e-94a7-8ba278d39525 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:23.571926 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:23.574190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276de098-bc97-4a5e-94a7-8ba278d39525 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:23.610234 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39feb0f-acc5-47e6-8ea1-856424df5b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:23.610896 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39feb0f-acc5-47e6-8ea1-856424df5b3b None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:23.611177 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39feb0f-acc5-47e6-8ea1-856424df5b3b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:23.611451 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39feb0f-acc5-47e6-8ea1-856424df5b3b None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:23.612082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39feb0f-acc5-47e6-8ea1-856424df5b3b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:23.612384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39feb0f-acc5-47e6-8ea1-856424df5b3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:23.614729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39feb0f-acc5-47e6-8ea1-856424df5b3b None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:23.619988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 437/886] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:33:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:23.629755 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f8325-a711-4f93-9e7d-aa399f473465 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:23.695991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:23.736031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f8325-a711-4f93-9e7d-aa399f473465 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:23.772098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541139d-3008-4c4c-9f1d-25e8dfb62f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:23.772740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541139d-3008-4c4c-9f1d-25e8dfb62f6e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:23.773025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541139d-3008-4c4c-9f1d-25e8dfb62f6e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:23.773297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541139d-3008-4c4c-9f1d-25e8dfb62f6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:23.773992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541139d-3008-4c4c-9f1d-25e8dfb62f6e None admin] RBAC: Adding query filter params (interface=public, service_id=e230fe1befdb4e99a25f35a6ce2451a5, region_id=RegionOne) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:23.774301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541139d-3008-4c4c-9f1d-25e8dfb62f6e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:23.775655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541139d-3008-4c4c-9f1d-25e8dfb62f6e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:23.781082 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 450/887] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 24 07:33:23 2022] GET /identity/v3/endpoints?service_id=e230fe1befdb4e99a25f35a6ce2451a5&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:23.786462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39feb0f-acc5-47e6-8ea1-856424df5b3b None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:23.892025 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39feb0f-acc5-47e6-8ea1-856424df5b3b None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:23.928079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64f752-c37d-4d63-abaa-02dcb702168a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:23.928720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64f752-c37d-4d63-abaa-02dcb702168a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:23.929006 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64f752-c37d-4d63-abaa-02dcb702168a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:23.929286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64f752-c37d-4d63-abaa-02dcb702168a None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:23.929892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64f752-c37d-4d63-abaa-02dcb702168a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:23.931252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64f752-c37d-4d63-abaa-02dcb702168a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:23.937324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 438/888] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 24 07:33:23 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:33:24.562504 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98552c80-da15-47e2-8165-a895a1927837 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:24.563048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98552c80-da15-47e2-8165-a895a1927837 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:24.563048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98552c80-da15-47e2-8165-a895a1927837 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:24.564653 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 451/889] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:24.570928 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08d4a3-a9d0-49d2-9a0c-ed627b9023ab None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:24.571211 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08d4a3-a9d0-49d2-9a0c-ed627b9023ab None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:24.571485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08d4a3-a9d0-49d2-9a0c-ed627b9023ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:24.590187 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-6e08d4a3-a9d0-49d2-9a0c-ed627b9023ab None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:24.591376 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6e08d4a3-a9d0-49d2-9a0c-ed627b9023ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:24.603572 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:24.663017 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 439/890] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:24.931611 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98552c80-da15-47e2-8165-a895a1927837 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:24.961388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f725b4b-7ce0-4fa1-a475-05c3c918f943 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:24.962017 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f725b4b-7ce0-4fa1-a475-05c3c918f943 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:24.962478 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f725b4b-7ce0-4fa1-a475-05c3c918f943 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:24.964267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 440/891] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:24.969317 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373a6f9-3619-43a7-bfc4-43e84e08be19 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:24.969603 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373a6f9-3619-43a7-bfc4-43e84e08be19 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:24.969877 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373a6f9-3619-43a7-bfc4-43e84e08be19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:24.991873 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-3373a6f9-3619-43a7-bfc4-43e84e08be19 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:24.993383 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-3373a6f9-3619-43a7-bfc4-43e84e08be19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:25.061447 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98552c80-da15-47e2-8165-a895a1927837 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:25.084373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 441/892] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:25.096558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:25.105835 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201e036-ed8c-4bf5-8fff-81284f7e6ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:25.106760 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201e036-ed8c-4bf5-8fff-81284f7e6ce8 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:25.107226 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201e036-ed8c-4bf5-8fff-81284f7e6ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:25.107698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201e036-ed8c-4bf5-8fff-81284f7e6ce8 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:25.108571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201e036-ed8c-4bf5-8fff-81284f7e6ce8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:25.110144 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201e036-ed8c-4bf5-8fff-81284f7e6ce8 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:25.161838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600edeb-d3dc-4d53-a56a-a953e70e4b94 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:25.162121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600edeb-d3dc-4d53-a56a-a953e70e4b94 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:25.162439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600edeb-d3dc-4d53-a56a-a953e70e4b94 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:25.163937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 442/893] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:25.168500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f16ca-4207-44eb-aa5f-73e66c7627b7 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:25.168827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f16ca-4207-44eb-aa5f-73e66c7627b7 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:25.169106 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f16ca-4207-44eb-aa5f-73e66c7627b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:25.179381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 452/894] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 24 07:33:24 2022] POST /identity/v3/services => generated 241 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:33:25.194656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-583f16ca-4207-44eb-aa5f-73e66c7627b7 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:25.196005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-583f16ca-4207-44eb-aa5f-73e66c7627b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:25.285499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 443/895] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:25.367948 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201e036-ed8c-4bf5-8fff-81284f7e6ce8 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:25.473648 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201e036-ed8c-4bf5-8fff-81284f7e6ce8 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:25.509289 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9218071-66db-470d-8c84-91fe8f1fefbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:25.510289 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9218071-66db-470d-8c84-91fe8f1fefbc None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:25.510767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9218071-66db-470d-8c84-91fe8f1fefbc None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:25.511236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9218071-66db-470d-8c84-91fe8f1fefbc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:25.522387 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9218071-66db-470d-8c84-91fe8f1fefbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-d9218071-66db-470d-8c84-91fe8f1fefbc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:25.527795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:25.528723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:25.529574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:25.529574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:25.529574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:25.529574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:25.529574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:25.529574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:25.529574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:25.529574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:25.529574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:25.529574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:25.529574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:33:25.529574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:25.536805 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 453/896] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Aug 24 07:33:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:25.539071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f16ca-4207-44eb-aa5f-73e66c7627b7 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:25.543597 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:25.569901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9218071-66db-470d-8c84-91fe8f1fefbc None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:25.637072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:25.667703 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f16ca-4207-44eb-aa5f-73e66c7627b7 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:25.697113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9218071-66db-470d-8c84-91fe8f1fefbc None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:25.711172 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097dfef-9a4b-42f6-82ac-9f4d913912dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:25.712147 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097dfef-9a4b-42f6-82ac-9f4d913912dd None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:25.712434 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097dfef-9a4b-42f6-82ac-9f4d913912dd None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:25.713006 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097dfef-9a4b-42f6-82ac-9f4d913912dd None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:25.713759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097dfef-9a4b-42f6-82ac-9f4d913912dd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:25.714067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097dfef-9a4b-42f6-82ac-9f4d913912dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:25.715974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097dfef-9a4b-42f6-82ac-9f4d913912dd None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:25.728810 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 444/897] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Aug 24 07:33:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:25.734589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097dfef-9a4b-42f6-82ac-9f4d913912dd None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:25.738932 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffe452-c93a-4f9c-8fa1-c7eea36e9c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:25.739656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffe452-c93a-4f9c-8fa1-c7eea36e9c04 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:25.740005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffe452-c93a-4f9c-8fa1-c7eea36e9c04 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:25.740346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffe452-c93a-4f9c-8fa1-c7eea36e9c04 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:25.741025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffe452-c93a-4f9c-8fa1-c7eea36e9c04 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:25.743697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffe452-c93a-4f9c-8fa1-c7eea36e9c04 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-8dffe452-c93a-4f9c-8fa1-c7eea36e9c04 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:25.752726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:25.754207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:25.755457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 454/898] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Aug 24 07:33:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:25.766203 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffe452-c93a-4f9c-8fa1-c7eea36e9c04 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:25.807331 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:25.852197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097dfef-9a4b-42f6-82ac-9f4d913912dd None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:25.888266 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffe452-c93a-4f9c-8fa1-c7eea36e9c04 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:25.888886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08dfbf-c4a9-4786-bddb-f096aa25097a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:25.889682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08dfbf-c4a9-4786-bddb-f096aa25097a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:25.890140 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08dfbf-c4a9-4786-bddb-f096aa25097a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:25.890610 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08dfbf-c4a9-4786-bddb-f096aa25097a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:25.899954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:25.903347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08dfbf-c4a9-4786-bddb-f096aa25097a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:25.905004 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08dfbf-c4a9-4786-bddb-f096aa25097a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-6c08dfbf-c4a9-4786-bddb-f096aa25097a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:25.920412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:25.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:25.922587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:25.922587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:25.922587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:25.924228 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 445/899] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:25.925517 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea528412-1906-4419-a50b-8380b11a479b None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:25.925914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea528412-1906-4419-a50b-8380b11a479b None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:25.926192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea528412-1906-4419-a50b-8380b11a479b None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:25.927545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 446/900] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:25.928353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad25fd-6740-420a-9f8e-e4fa4736f981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:25.929203 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad25fd-6740-420a-9f8e-e4fa4736f981 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:25.929678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad25fd-6740-420a-9f8e-e4fa4736f981 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:25.929951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad25fd-6740-420a-9f8e-e4fa4736f981 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:25.930621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad25fd-6740-420a-9f8e-e4fa4736f981 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:25.930934 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad25fd-6740-420a-9f8e-e4fa4736f981 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:25.932316 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea528412-1906-4419-a50b-8380b11a479b None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:25.932586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad25fd-6740-420a-9f8e-e4fa4736f981 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:25.941422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 455/901] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 24 07:33:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:25.941422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d3e4-7e11-4636-929d-845f86416e5e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:25.941422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d3e4-7e11-4636-929d-845f86416e5e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:25.941422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d3e4-7e11-4636-929d-845f86416e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:25.957608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-4d03d3e4-7e11-4636-929d-845f86416e5e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:25.958803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4d03d3e4-7e11-4636-929d-845f86416e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:26.026394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 456/902] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:26.026967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03d3e4-7e11-4636-929d-845f86416e5e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:26.052559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea528412-1906-4419-a50b-8380b11a479b None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:26.071372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:26.090670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9860cf9-3908-4ca9-8f53-939f76769fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:26.091850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9860cf9-3908-4ca9-8f53-939f76769fc4 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:26.092006 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9860cf9-3908-4ca9-8f53-939f76769fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:26.092417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9860cf9-3908-4ca9-8f53-939f76769fc4 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:26.093266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9860cf9-3908-4ca9-8f53-939f76769fc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:26.093776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9860cf9-3908-4ca9-8f53-939f76769fc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:26.095298 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9860cf9-3908-4ca9-8f53-939f76769fc4 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:26.109556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 447/903] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:26.118619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9860cf9-3908-4ca9-8f53-939f76769fc4 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:26.150589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03d3e4-7e11-4636-929d-845f86416e5e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:26.154219 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:26.192698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ef0a9-ed5b-4218-a8fc-57ec8850bd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:26.192698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ef0a9-ed5b-4218-a8fc-57ec8850bd88 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:26.192698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ef0a9-ed5b-4218-a8fc-57ec8850bd88 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:26.193051 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ef0a9-ed5b-4218-a8fc-57ec8850bd88 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:26.193778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ef0a9-ed5b-4218-a8fc-57ec8850bd88 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:26.196781 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ef0a9-ed5b-4218-a8fc-57ec8850bd88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:26.196781 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ef0a9-ed5b-4218-a8fc-57ec8850bd88 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:26.207787 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 457/904] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 24 07:33:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:26.209357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ef0a9-ed5b-4218-a8fc-57ec8850bd88 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:26.244851 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9860cf9-3908-4ca9-8f53-939f76769fc4 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:26.283626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11aff8-6030-48f2-8a43-199ff34057f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:26.284115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11aff8-6030-48f2-8a43-199ff34057f7 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:26.284953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11aff8-6030-48f2-8a43-199ff34057f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:26.285363 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11aff8-6030-48f2-8a43-199ff34057f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:26.294620 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11aff8-6030-48f2-8a43-199ff34057f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:26.296065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11aff8-6030-48f2-8a43-199ff34057f7 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-db11aff8-6030-48f2-8a43-199ff34057f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:26.309277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:26.310324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:26.311278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:26.311278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:26.311278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:26.311278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:26.311278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:26.311278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:26.311278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:26.311278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:26.311278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:33:26.311278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:26.311278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 448/905] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 24 07:33:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:26.311858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11aff8-6030-48f2-8a43-199ff34057f7 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:26.343943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ef0a9-ed5b-4218-a8fc-57ec8850bd88 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:26.357494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:26.379800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cd8ef-f028-4710-a09c-139ab20e9430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:26.380414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cd8ef-f028-4710-a09c-139ab20e9430 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:26.380723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cd8ef-f028-4710-a09c-139ab20e9430 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:26.380997 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cd8ef-f028-4710-a09c-139ab20e9430 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:26.381693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356cd8ef-f028-4710-a09c-139ab20e9430 None admin] RBAC: Adding query filter params (interface=public, service_id=5cfefcc04df544c18013109040b60146, region_id=RegionOne) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:26.382001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356cd8ef-f028-4710-a09c-139ab20e9430 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:26.383360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356cd8ef-f028-4710-a09c-139ab20e9430 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:26.392982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 458/906] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 24 07:33:26 2022] GET /identity/v3/endpoints?service_id=5cfefcc04df544c18013109040b60146&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:26.392982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cd8ef-f028-4710-a09c-139ab20e9430 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:26.424157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:26.431942 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11aff8-6030-48f2-8a43-199ff34057f7 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:26.467732 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9eb126-b209-4e80-aa00-b1e8cc3e4781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:26.468387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9eb126-b209-4e80-aa00-b1e8cc3e4781 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:26.468698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9eb126-b209-4e80-aa00-b1e8cc3e4781 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:26.468974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9eb126-b209-4e80-aa00-b1e8cc3e4781 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:26.469562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9eb126-b209-4e80-aa00-b1e8cc3e4781 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:26.470927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9eb126-b209-4e80-aa00-b1e8cc3e4781 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-ce9eb126-b209-4e80-aa00-b1e8cc3e4781 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:26.480874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:26.481838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:26.482692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:26.482692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:26.482692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:26.482692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:26.482692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:26.482692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:26.482692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:26.482692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:26.482692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:26.482692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:26.482692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:26.482692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:26.482692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:33:26.482692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:26.486765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 449/907] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:33:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:26.487626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9eb126-b209-4e80-aa00-b1e8cc3e4781 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:26.495955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cd8ef-f028-4710-a09c-139ab20e9430 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:26.535649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2106f-2b5e-4a96-9251-776154972bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:26.536205 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2106f-2b5e-4a96-9251-776154972bba None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:26.536488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2106f-2b5e-4a96-9251-776154972bba None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:26.536786 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2106f-2b5e-4a96-9251-776154972bba None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:26.537517 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2106f-2b5e-4a96-9251-776154972bba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:26.537821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2106f-2b5e-4a96-9251-776154972bba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:26.539349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2106f-2b5e-4a96-9251-776154972bba None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:26.556713 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 459/908] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Aug 24 07:33:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:26.557541 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2106f-2b5e-4a96-9251-776154972bba None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:26.603258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:26.606339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9eb126-b209-4e80-aa00-b1e8cc3e4781 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:26.642446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ad095-76e2-4c65-8a96-1135e0a18437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:26.643043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ad095-76e2-4c65-8a96-1135e0a18437 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:26.643320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ad095-76e2-4c65-8a96-1135e0a18437 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:26.643591 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ad095-76e2-4c65-8a96-1135e0a18437 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:26.644972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ad095-76e2-4c65-8a96-1135e0a18437 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:26.646339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ad095-76e2-4c65-8a96-1135e0a18437 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:26.660679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 450/909] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 24 07:33:26 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:33:26.660679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ad095-76e2-4c65-8a96-1135e0a18437 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:26.665044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:26.676484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2106f-2b5e-4a96-9251-776154972bba None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:26.720959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3951e-8890-46f7-b0d3-6a350d6528f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:26.721727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3951e-8890-46f7-b0d3-6a350d6528f7 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:26.723038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3951e-8890-46f7-b0d3-6a350d6528f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:26.723488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3951e-8890-46f7-b0d3-6a350d6528f7 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:26.724325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3951e-8890-46f7-b0d3-6a350d6528f7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:26.725197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3951e-8890-46f7-b0d3-6a350d6528f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:26.726460 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3951e-8890-46f7-b0d3-6a350d6528f7 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:26.737376 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 460/910] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:33:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:26.738785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3951e-8890-46f7-b0d3-6a350d6528f7 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:26.803601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ad095-76e2-4c65-8a96-1135e0a18437 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:26.848069 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe14bca-a639-414d-adf0-e336680f5e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:26.848700 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe14bca-a639-414d-adf0-e336680f5e7f None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:26.848985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe14bca-a639-414d-adf0-e336680f5e7f None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:26.849261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe14bca-a639-414d-adf0-e336680f5e7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:26.866739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3951e-8890-46f7-b0d3-6a350d6528f7 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:26.881943 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:26.910568 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483611da-c8ed-4129-bc2a-297f4de03117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:26.911510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483611da-c8ed-4129-bc2a-297f4de03117 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:26.912056 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483611da-c8ed-4129-bc2a-297f4de03117 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:26.912336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483611da-c8ed-4129-bc2a-297f4de03117 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:26.913147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483611da-c8ed-4129-bc2a-297f4de03117 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:26.913456 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483611da-c8ed-4129-bc2a-297f4de03117 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:26.915382 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483611da-c8ed-4129-bc2a-297f4de03117 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:26.921396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe14bca-a639-414d-adf0-e336680f5e7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:26.921811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 461/911] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 24 07:33:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:26.923003 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe14bca-a639-414d-adf0-e336680f5e7f None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:26.928519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483611da-c8ed-4129-bc2a-297f4de03117 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:26.957214 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:27.043969 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 451/912] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:26 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:33:27.052697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe14bca-a639-414d-adf0-e336680f5e7f None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:27.071357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483611da-c8ed-4129-bc2a-297f4de03117 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:27.107184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d350c49-00cd-4d5c-acf0-2880e08836a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:27.107904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d350c49-00cd-4d5c-acf0-2880e08836a3 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:27.108188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d350c49-00cd-4d5c-acf0-2880e08836a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:27.108459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d350c49-00cd-4d5c-acf0-2880e08836a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:27.109159 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d350c49-00cd-4d5c-acf0-2880e08836a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:27.110622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d350c49-00cd-4d5c-acf0-2880e08836a3 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:27.130063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:27.167761 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe14bca-a639-414d-adf0-e336680f5e7f None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:27.190143 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 462/913] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:33:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:33:27.198266 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cf5ec-76eb-4fbc-8092-f7ec39b3b502 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:27.198610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cf5ec-76eb-4fbc-8092-f7ec39b3b502 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:27.198900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cf5ec-76eb-4fbc-8092-f7ec39b3b502 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:27.200244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 463/914] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:27.206058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc7f57-0f11-4da6-9c03-0fbd61a705ad None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:27.206372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc7f57-0f11-4da6-9c03-0fbd61a705ad None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:27.206651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc7f57-0f11-4da6-9c03-0fbd61a705ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:27.211388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513448c-ebf1-4fb3-bee2-7045edbbfb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:27.212508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513448c-ebf1-4fb3-bee2-7045edbbfb83 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:27.212978 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513448c-ebf1-4fb3-bee2-7045edbbfb83 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:27.214146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513448c-ebf1-4fb3-bee2-7045edbbfb83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:27.228018 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513448c-ebf1-4fb3-bee2-7045edbbfb83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:27.229534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513448c-ebf1-4fb3-bee2-7045edbbfb83 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:27.239754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-1dbc7f57-0f11-4da6-9c03-0fbd61a705ad None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:27.241781 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-1dbc7f57-0f11-4da6-9c03-0fbd61a705ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-c513448c-ebf1-4fb3-bee2-7045edbbfb83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:27.246303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:27.247373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:27.248292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:27.248292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:27.248292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:27.248292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 452/915] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:27.254250 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513448c-ebf1-4fb3-bee2-7045edbbfb83 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:27.340914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 464/916] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:27.389330 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513448c-ebf1-4fb3-bee2-7045edbbfb83 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:27.403213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:27.426061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de33447-9626-44bd-8fb1-e3892dedac39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:27.426665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de33447-9626-44bd-8fb1-e3892dedac39 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:27.426943 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de33447-9626-44bd-8fb1-e3892dedac39 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:27.427217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de33447-9626-44bd-8fb1-e3892dedac39 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:27.427852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de33447-9626-44bd-8fb1-e3892dedac39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:27.428156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de33447-9626-44bd-8fb1-e3892dedac39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:27.429550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de33447-9626-44bd-8fb1-e3892dedac39 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:27.442060 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 453/917] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:27.447811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc7f57-0f11-4da6-9c03-0fbd61a705ad None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:27.452287 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:27.551793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc7f57-0f11-4da6-9c03-0fbd61a705ad None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:27.586674 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d55f81-a502-489a-a137-4bcaef54761b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:27.587300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d55f81-a502-489a-a137-4bcaef54761b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:27.587575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d55f81-a502-489a-a137-4bcaef54761b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:27.587846 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d55f81-a502-489a-a137-4bcaef54761b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:27.595151 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d55f81-a502-489a-a137-4bcaef54761b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:27.596658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d55f81-a502-489a-a137-4bcaef54761b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-64d55f81-a502-489a-a137-4bcaef54761b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:27.608472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:27.609385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:27.610274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:27.610274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:27.610274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:27.610274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:27.610274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:27.610274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:27.610274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:27.610274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:27.610274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 465/918] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:27.617856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de33447-9626-44bd-8fb1-e3892dedac39 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:27.647720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d55f81-a502-489a-a137-4bcaef54761b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:27.712364 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:27.727726 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de33447-9626-44bd-8fb1-e3892dedac39 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:27.753765 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d55f81-a502-489a-a137-4bcaef54761b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:27.765884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a749ba-a946-4e7e-a922-c31948f8c06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:27.766502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a749ba-a946-4e7e-a922-c31948f8c06c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:27.766780 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a749ba-a946-4e7e-a922-c31948f8c06c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:27.767055 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a749ba-a946-4e7e-a922-c31948f8c06c None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:27.767746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a749ba-a946-4e7e-a922-c31948f8c06c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:27.768052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a749ba-a946-4e7e-a922-c31948f8c06c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:27.769545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a749ba-a946-4e7e-a922-c31948f8c06c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:27.781498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 454/919] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:27.787853 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a749ba-a946-4e7e-a922-c31948f8c06c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:27.794033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfa7f4-36a0-4a64-b439-bc2ebd53241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:27.794952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfa7f4-36a0-4a64-b439-bc2ebd53241d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:27.795436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfa7f4-36a0-4a64-b439-bc2ebd53241d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:27.795913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfa7f4-36a0-4a64-b439-bc2ebd53241d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:27.796733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cfa7f4-36a0-4a64-b439-bc2ebd53241d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:27.798700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cfa7f4-36a0-4a64-b439-bc2ebd53241d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-01cfa7f4-36a0-4a64-b439-bc2ebd53241d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:27.810195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:27.811232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:27.812062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 466/920] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 24 07:33:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:27.819687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfa7f4-36a0-4a64-b439-bc2ebd53241d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:27.848735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:27.905063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a749ba-a946-4e7e-a922-c31948f8c06c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:27.953588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa38877-c249-40ac-8fb3-5275b499c7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:27.954682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa38877-c249-40ac-8fb3-5275b499c7ab None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:27.954939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfa7f4-36a0-4a64-b439-bc2ebd53241d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:27.955404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa38877-c249-40ac-8fb3-5275b499c7ab None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:27.955893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa38877-c249-40ac-8fb3-5275b499c7ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:27.957526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa38877-c249-40ac-8fb3-5275b499c7ab None admin] RBAC: Adding query filter params (role.id=f126e5eef4964aa5af83192f79e8a1dd, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=be379e1a6add42b69e9fdd676efca280) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:27.958081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa38877-c249-40ac-8fb3-5275b499c7ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:27.959730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa38877-c249-40ac-8fb3-5275b499c7ab None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:27.966211 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 455/921] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:27 2022] GET /identity/v3/role_assignments?user.id=be379e1a6add42b69e9fdd676efca280&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=f126e5eef4964aa5af83192f79e8a1dd => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:27.993060 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:28.001277 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6164860-7a52-463e-96aa-1b7adc6d4754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:28.001901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6164860-7a52-463e-96aa-1b7adc6d4754 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:28.002182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6164860-7a52-463e-96aa-1b7adc6d4754 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:28.002455 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6164860-7a52-463e-96aa-1b7adc6d4754 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:28.003694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6164860-7a52-463e-96aa-1b7adc6d4754 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:28.004001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6164860-7a52-463e-96aa-1b7adc6d4754 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:28.005721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6164860-7a52-463e-96aa-1b7adc6d4754 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:28.011326 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 467/922] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 24 07:33:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:28.017402 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa38877-c249-40ac-8fb3-5275b499c7ab None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:28.125697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:28.132844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa38877-c249-40ac-8fb3-5275b499c7ab None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:28.172864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cb3d6-f762-4cda-b7a9-753f08b95c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:28.173714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cb3d6-f762-4cda-b7a9-753f08b95c8d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:28.174182 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cb3d6-f762-4cda-b7a9-753f08b95c8d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:28.174661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cb3d6-f762-4cda-b7a9-753f08b95c8d None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:28.175706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198cb3d6-f762-4cda-b7a9-753f08b95c8d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:28.176157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198cb3d6-f762-4cda-b7a9-753f08b95c8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:28.177743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198cb3d6-f762-4cda-b7a9-753f08b95c8d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:28.183498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 456/923] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 24 07:33:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:28.189559 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6164860-7a52-463e-96aa-1b7adc6d4754 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:28.294187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6164860-7a52-463e-96aa-1b7adc6d4754 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:28.335078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9b24c-045a-45bd-8dc0-17ed14996638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:28.335697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9b24c-045a-45bd-8dc0-17ed14996638 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:28.335977 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9b24c-045a-45bd-8dc0-17ed14996638 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:28.336254 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9b24c-045a-45bd-8dc0-17ed14996638 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:28.336982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9b24c-045a-45bd-8dc0-17ed14996638 None admin] RBAC: Adding query filter params (interface=public, service_id=14835d27845f47f3a2ac8bc628094dcc, region_id=RegionOne) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:28.337293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9b24c-045a-45bd-8dc0-17ed14996638 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:28.338657 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9b24c-045a-45bd-8dc0-17ed14996638 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:28.344392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 468/924] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 24 07:33:28 2022] GET /identity/v3/endpoints?service_id=14835d27845f47f3a2ac8bc628094dcc&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:28.350163 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cb3d6-f762-4cda-b7a9-753f08b95c8d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:28.461732 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cb3d6-f762-4cda-b7a9-753f08b95c8d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:28.497867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef1935-22f0-4b9a-a579-ea7894ef80d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:28.498484 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef1935-22f0-4b9a-a579-ea7894ef80d8 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:28.498767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef1935-22f0-4b9a-a579-ea7894ef80d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:28.499046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef1935-22f0-4b9a-a579-ea7894ef80d8 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:28.499650 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef1935-22f0-4b9a-a579-ea7894ef80d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:28.501093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef1935-22f0-4b9a-a579-ea7894ef80d8 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:28.508367 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 457/925] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 24 07:33:28 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:33:28.689863 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080d85e-53e6-4fdc-beeb-14f385087f2e None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:28.690192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080d85e-53e6-4fdc-beeb-14f385087f2e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:28.690423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080d85e-53e6-4fdc-beeb-14f385087f2e None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:28.692174 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 469/926] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:28.697593 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fabb2-b6cf-4412-8673-a9526524843c None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:28.697885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fabb2-b6cf-4412-8673-a9526524843c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:28.698165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fabb2-b6cf-4412-8673-a9526524843c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:28.717087 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-cc6fabb2-b6cf-4412-8673-a9526524843c None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:28.718207 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-cc6fabb2-b6cf-4412-8673-a9526524843c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:28.737293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:28.789213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 458/927] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:29.059510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080d85e-53e6-4fdc-beeb-14f385087f2e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:29.120052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:29.166034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080d85e-53e6-4fdc-beeb-14f385087f2e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:29.202676 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e54df-b0bd-4dfe-81e0-010304c7868d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:29.203311 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e54df-b0bd-4dfe-81e0-010304c7868d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:29.203593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e54df-b0bd-4dfe-81e0-010304c7868d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:29.203866 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e54df-b0bd-4dfe-81e0-010304c7868d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:29.204451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e54df-b0bd-4dfe-81e0-010304c7868d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:29.205838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e54df-b0bd-4dfe-81e0-010304c7868d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-d03e54df-b0bd-4dfe-81e0-010304c7868d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:29.214581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:29.215594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:29.216549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 470/928] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Aug 24 07:33:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:29.222667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6fabb2-b6cf-4412-8673-a9526524843c None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:29.289271 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25489d51-ed41-4e76-9ec8-971b838eb941 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:29.289602 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25489d51-ed41-4e76-9ec8-971b838eb941 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:29.289857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25489d51-ed41-4e76-9ec8-971b838eb941 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:29.292366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 471/929] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:29.296284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3f6fe-4511-4cc0-99be-6bd7ebd535b9 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:29.296566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3f6fe-4511-4cc0-99be-6bd7ebd535b9 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:29.296881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3f6fe-4511-4cc0-99be-6bd7ebd535b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:29.315284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-39d3f6fe-4511-4cc0-99be-6bd7ebd535b9 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:29.317433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-39d3f6fe-4511-4cc0-99be-6bd7ebd535b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:29.331855 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6fabb2-b6cf-4412-8673-a9526524843c None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:29.379096 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67053ff2-ef3c-4441-8121-ddbbeac5f1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:29.379984 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67053ff2-ef3c-4441-8121-ddbbeac5f1ac None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:29.380266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67053ff2-ef3c-4441-8121-ddbbeac5f1ac None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:29.380539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67053ff2-ef3c-4441-8121-ddbbeac5f1ac None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:29.381644 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67053ff2-ef3c-4441-8121-ddbbeac5f1ac None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:29.381971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67053ff2-ef3c-4441-8121-ddbbeac5f1ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:29.383765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67053ff2-ef3c-4441-8121-ddbbeac5f1ac None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:29.389170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 459/931] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 24 07:33:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:29.390172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 472/931] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:29.395309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67053ff2-ef3c-4441-8121-ddbbeac5f1ac None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:29.414235 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:29.498211 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67053ff2-ef3c-4441-8121-ddbbeac5f1ac None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:29.536670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c28ddd-de66-4f0c-ac55-995b8233062c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:29.536670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c28ddd-de66-4f0c-ac55-995b8233062c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:29.536670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c28ddd-de66-4f0c-ac55-995b8233062c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:29.536670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c28ddd-de66-4f0c-ac55-995b8233062c None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:29.537063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c28ddd-de66-4f0c-ac55-995b8233062c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:29.537063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c28ddd-de66-4f0c-ac55-995b8233062c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:29.538348 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c28ddd-de66-4f0c-ac55-995b8233062c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:29.543782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 460/932] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 24 07:33:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:29.549402 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3f6fe-4511-4cc0-99be-6bd7ebd535b9 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:29.604763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:29.653713 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3f6fe-4511-4cc0-99be-6bd7ebd535b9 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:29.666777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c28ddd-de66-4f0c-ac55-995b8233062c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:29.702808 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663eed86-8c90-4b67-a173-395943a2aa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:29.703423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663eed86-8c90-4b67-a173-395943a2aa12 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:29.703713 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663eed86-8c90-4b67-a173-395943a2aa12 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:29.703988 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663eed86-8c90-4b67-a173-395943a2aa12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:29.704687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663eed86-8c90-4b67-a173-395943a2aa12 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:29.706904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663eed86-8c90-4b67-a173-395943a2aa12 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:29.759530 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:29.777459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c28ddd-de66-4f0c-ac55-995b8233062c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:29.793214 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 473/933] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:33:29 2022] POST /identity/v3/endpoints => generated 347 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:33:29.827086 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b69af-d02b-4b16-be3a-d9ce37916b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:29.827701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b69af-d02b-4b16-be3a-d9ce37916b77 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:29.827995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b69af-d02b-4b16-be3a-d9ce37916b77 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:29.828261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b69af-d02b-4b16-be3a-d9ce37916b77 None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:29.829342 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b69af-d02b-4b16-be3a-d9ce37916b77 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:29.831271 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b69af-d02b-4b16-be3a-d9ce37916b77 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:29.834722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: WARNING keystone.server.flask.common [None req-be5b69af-d02b-4b16-be3a-d9ce37916b77 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 07:33:29.886549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.sql.core [None req-be5b69af-d02b-4b16-be3a-d9ce37916b77 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 07:33:29.886549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 07:33:29.886549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [parameters: {'id': '66ce7ef4511a4fe88542a53cdadb548d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 07:33:29.886549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88250) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 07:33:29.895156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-be5b69af-d02b-4b16-be3a-d9ce37916b77 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 66ce7ef4511a4fe88542a53cdadb548d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 66ce7ef4511a4fe88542a53cdadb548d. Aug 24 07:33:29.895156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:29.895156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:33:29.895156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:33:29.895156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:33:29.895156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:33:29.895156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:33:29.895156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:33:29.895156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:33:29.895156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:33:29.895156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:33:29.895156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:33:29.895156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application result.read() Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 07:33:29.896185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application yield resource Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise self.value Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application session.commit() Aug 24 07:33:29.897206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self.session.flush() Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application compat.raise_( Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 07:33:29.898109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 07:33:29.898995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application result.read() Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:33:29.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [parameters: {'id': '66ce7ef4511a4fe88542a53cdadb548d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:29.901852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:29.903937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:29.906134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:29.906134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 07:33:29.906134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 07:33:29.906134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:29.906134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:29.906134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 07:33:29.906134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 07:33:29.906134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 66ce7ef4511a4fe88542a53cdadb548d. Aug 24 07:33:29.906134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:29.906134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 461/934] 10.222.0.26 () {60 vars in 1140 bytes} [Wed Aug 24 07:33:29 2022] POST /identity/v3/projects => generated 262 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:29.909543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663eed86-8c90-4b67-a173-395943a2aa12 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:30.018831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663eed86-8c90-4b67-a173-395943a2aa12 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:30.033733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:30.054701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1333d3-8d60-4a08-9535-79b7d4621adf None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:30.055189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1333d3-8d60-4a08-9535-79b7d4621adf None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:30.055500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1333d3-8d60-4a08-9535-79b7d4621adf None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:30.056508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3530b-5173-4c0a-92d8-df78152e9454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:30.057263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3530b-5173-4c0a-92d8-df78152e9454 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:30.057645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3530b-5173-4c0a-92d8-df78152e9454 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:30.058021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3530b-5173-4c0a-92d8-df78152e9454 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:30.059793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 462/935] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:30.065112 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3530b-5173-4c0a-92d8-df78152e9454 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:30.067294 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3530b-5173-4c0a-92d8-df78152e9454 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:30.070124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b603c6b-2327-41db-bce4-baee437adb21 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:30.070413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b603c6b-2327-41db-bce4-baee437adb21 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:30.070742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b603c6b-2327-41db-bce4-baee437adb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-b2f3530b-5173-4c0a-92d8-df78152e9454 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:30.083907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:30.089722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:30.090630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:30.090630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:30.090630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:30.090630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:30.090630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:30.090630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:30.090630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:30.090630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:30.090630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 474/936] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:30.092916 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3530b-5173-4c0a-92d8-df78152e9454 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:30.093110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-6b603c6b-2327-41db-bce4-baee437adb21 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:30.094671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6b603c6b-2327-41db-bce4-baee437adb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:30.190089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 463/937] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:30.203625 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3530b-5173-4c0a-92d8-df78152e9454 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:30.240142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d97b73-71ba-4a0e-a01a-d2b4a6a7bcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:30.240776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d97b73-71ba-4a0e-a01a-d2b4a6a7bcd1 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:30.241058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d97b73-71ba-4a0e-a01a-d2b4a6a7bcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:30.241333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d97b73-71ba-4a0e-a01a-d2b4a6a7bcd1 None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:30.241996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d97b73-71ba-4a0e-a01a-d2b4a6a7bcd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:30.242300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d97b73-71ba-4a0e-a01a-d2b4a6a7bcd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:30.243776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d97b73-71ba-4a0e-a01a-d2b4a6a7bcd1 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:30.269686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 475/938] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 24 07:33:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:30.469378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09198d50-4256-4d93-b222-dcea729d50b0 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:30.469891 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09198d50-4256-4d93-b222-dcea729d50b0 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:30.469891 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09198d50-4256-4d93-b222-dcea729d50b0 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:30.471192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 464/939] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:30.471616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d97b73-71ba-4a0e-a01a-d2b4a6a7bcd1 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:30.477268 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228aaab-8327-4c8b-babf-f527c82c2252 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:30.477555 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228aaab-8327-4c8b-babf-f527c82c2252 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:30.477866 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228aaab-8327-4c8b-babf-f527c82c2252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:30.499638 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-6228aaab-8327-4c8b-babf-f527c82c2252 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:30.501452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6228aaab-8327-4c8b-babf-f527c82c2252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:30.501564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:30.583299 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d97b73-71ba-4a0e-a01a-d2b4a6a7bcd1 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:30.590489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:30.606423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 465/940] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:30.632065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c26f7-4c98-4152-a430-b5ea7566b451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:30.633095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c26f7-4c98-4152-a430-b5ea7566b451 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:30.633635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c26f7-4c98-4152-a430-b5ea7566b451 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:30.634117 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c26f7-4c98-4152-a430-b5ea7566b451 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:30.646570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c26f7-4c98-4152-a430-b5ea7566b451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-708c26f7-4c98-4152-a430-b5ea7566b451 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:30.651569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:30.652769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:30.653672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:30.653672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:30.653672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:30.653672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:30.653672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:30.653672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:30.653672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:30.653672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:30.653672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:30.653672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:30.653672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:33:30.653672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:30.656340 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 476/941] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Aug 24 07:33:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:30.662402 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228aaab-8327-4c8b-babf-f527c82c2252 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:30.762953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228aaab-8327-4c8b-babf-f527c82c2252 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:30.798868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e720157-199d-4072-a8ad-34b71f973416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:30.799471 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e720157-199d-4072-a8ad-34b71f973416 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:30.799749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e720157-199d-4072-a8ad-34b71f973416 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:30.800578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e720157-199d-4072-a8ad-34b71f973416 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:30.801363 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e720157-199d-4072-a8ad-34b71f973416 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:30.801668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e720157-199d-4072-a8ad-34b71f973416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:30.803152 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e720157-199d-4072-a8ad-34b71f973416 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:30.811578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 466/942] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Aug 24 07:33:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:30.817300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c26f7-4c98-4152-a430-b5ea7566b451 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:30.885681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e720157-199d-4072-a8ad-34b71f973416 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:30.922924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c26f7-4c98-4152-a430-b5ea7566b451 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:30.941788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:30.962571 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:30.966298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21f22a-e161-4db9-9653-173c5e587bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:30.967445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21f22a-e161-4db9-9653-173c5e587bda None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:30.968124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21f22a-e161-4db9-9653-173c5e587bda None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:30.969059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21f22a-e161-4db9-9653-173c5e587bda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:30.983653 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21f22a-e161-4db9-9653-173c5e587bda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:30.996439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21f22a-e161-4db9-9653-173c5e587bda None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-cd21f22a-e161-4db9-9653-173c5e587bda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:31.005994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:31.007037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:31.007968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:31.007968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:31.007968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:31.010408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 477/943] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:31.012751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e720157-199d-4072-a8ad-34b71f973416 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:31.017061 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21f22a-e161-4db9-9653-173c5e587bda None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:31.059361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc45da-f209-4ae4-bed4-1bb86a753fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:31.060946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc45da-f209-4ae4-bed4-1bb86a753fd8 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:31.061414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc45da-f209-4ae4-bed4-1bb86a753fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:31.062107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc45da-f209-4ae4-bed4-1bb86a753fd8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:31.062890 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc45da-f209-4ae4-bed4-1bb86a753fd8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:31.066211 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc45da-f209-4ae4-bed4-1bb86a753fd8 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-51cc45da-f209-4ae4-bed4-1bb86a753fd8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:31.077137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:31.078294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:31.079190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:31.079190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:31.079190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:31.079190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:31.079190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:31.079190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:31.079190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:31.079190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:31.079190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:31.079190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:31.079190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:31.079190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:31.079190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:31.079190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:31.087908 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 467/944] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 24 07:33:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:31.089718 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc45da-f209-4ae4-bed4-1bb86a753fd8 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:31.146181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21f22a-e161-4db9-9653-173c5e587bda None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:31.183658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d6745-32e9-414f-967e-edcb966171c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:31.184639 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d6745-32e9-414f-967e-edcb966171c8 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:31.185118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d6745-32e9-414f-967e-edcb966171c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:31.185547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d6745-32e9-414f-967e-edcb966171c8 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:31.186340 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d6745-32e9-414f-967e-edcb966171c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:31.186896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d6745-32e9-414f-967e-edcb966171c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:31.188508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d6745-32e9-414f-967e-edcb966171c8 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:31.193656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc45da-f209-4ae4-bed4-1bb86a753fd8 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:31.197813 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:31.204476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 478/945] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:31.210167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d6745-32e9-414f-967e-edcb966171c8 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:31.219482 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:31.234942 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c517a4-e0b0-4b27-b2f4-b39030db45d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:31.235628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c517a4-e0b0-4b27-b2f4-b39030db45d4 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:31.235993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c517a4-e0b0-4b27-b2f4-b39030db45d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:31.236335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c517a4-e0b0-4b27-b2f4-b39030db45d4 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:31.237060 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c517a4-e0b0-4b27-b2f4-b39030db45d4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:31.237953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c517a4-e0b0-4b27-b2f4-b39030db45d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:31.239402 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c517a4-e0b0-4b27-b2f4-b39030db45d4 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:31.249094 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 468/946] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 24 07:33:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:31.251138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c517a4-e0b0-4b27-b2f4-b39030db45d4 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:31.343554 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d6745-32e9-414f-967e-edcb966171c8 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:31.363982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c517a4-e0b0-4b27-b2f4-b39030db45d4 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:31.391068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eeaceb-2118-4350-bd68-46cb4c4999d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:31.392139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eeaceb-2118-4350-bd68-46cb4c4999d1 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:31.392426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eeaceb-2118-4350-bd68-46cb4c4999d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:31.392874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eeaceb-2118-4350-bd68-46cb4c4999d1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:31.400801 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7485d88-4680-46e6-b98a-6a37ca1f909c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:31.401539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7485d88-4680-46e6-b98a-6a37ca1f909c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:31.401924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7485d88-4680-46e6-b98a-6a37ca1f909c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:31.402304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7485d88-4680-46e6-b98a-6a37ca1f909c None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:31.402430 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eeaceb-2118-4350-bd68-46cb4c4999d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:31.403133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7485d88-4680-46e6-b98a-6a37ca1f909c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:31.403538 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7485d88-4680-46e6-b98a-6a37ca1f909c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:31.403868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eeaceb-2118-4350-bd68-46cb4c4999d1 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:31.405120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7485d88-4680-46e6-b98a-6a37ca1f909c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:31.411508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 469/947] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 24 07:33:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:31.417118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7485d88-4680-46e6-b98a-6a37ca1f909c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-02eeaceb-2118-4350-bd68-46cb4c4999d1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:31.417366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:31.418297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:31.419831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:31.419831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:31.419831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:31.419831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:31.419831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:31.419831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:31.419831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:31.419831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:31.419831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:33:31.419831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:31.423603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 479/948] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 24 07:33:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:31.425613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eeaceb-2118-4350-bd68-46cb4c4999d1 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:31.478700 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:31.506170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:31.542883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7485d88-4680-46e6-b98a-6a37ca1f909c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:31.559348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eeaceb-2118-4350-bd68-46cb4c4999d1 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:31.588830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c4b30-ea33-483b-9d98-a867eabfee16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:31.589734 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c4b30-ea33-483b-9d98-a867eabfee16 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:31.590024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c4b30-ea33-483b-9d98-a867eabfee16 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:31.590298 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c4b30-ea33-483b-9d98-a867eabfee16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:31.590956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c4b30-ea33-483b-9d98-a867eabfee16 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:31.592494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c4b30-ea33-483b-9d98-a867eabfee16 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:31.603194 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951348a-5a08-4195-b255-c49a25c01369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:31.604102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951348a-5a08-4195-b255-c49a25c01369 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:31.604605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951348a-5a08-4195-b255-c49a25c01369 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:31.605113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951348a-5a08-4195-b255-c49a25c01369 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:31.606063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951348a-5a08-4195-b255-c49a25c01369 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:31.606631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951348a-5a08-4195-b255-c49a25c01369 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:31.608356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951348a-5a08-4195-b255-c49a25c01369 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:31.626930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 480/949] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Aug 24 07:33:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:31.633148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951348a-5a08-4195-b255-c49a25c01369 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:31.700728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 470/950] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:33:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:33:31.756862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951348a-5a08-4195-b255-c49a25c01369 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:31.776864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:31.804192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13080b-50ef-48a2-b168-d46145f0eef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:31.805273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13080b-50ef-48a2-b168-d46145f0eef3 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:31.805762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13080b-50ef-48a2-b168-d46145f0eef3 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:31.806258 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13080b-50ef-48a2-b168-d46145f0eef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:31.818907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13080b-50ef-48a2-b168-d46145f0eef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:31.820821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13080b-50ef-48a2-b168-d46145f0eef3 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-3a13080b-50ef-48a2-b168-d46145f0eef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:31.837769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:31.838807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:31.839708 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:31.839708 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:31.839708 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:31.842060 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 481/951] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:31.852771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c4b30-ea33-483b-9d98-a867eabfee16 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:31.876230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cf1d2-3fc1-4bb7-9693-70a42a35aa56 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:31.876796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cf1d2-3fc1-4bb7-9693-70a42a35aa56 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:31.877223 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cf1d2-3fc1-4bb7-9693-70a42a35aa56 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:31.879548 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 482/952] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:31.885010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15126acb-e601-4c95-97cb-4bcf8c179323 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:31.885447 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15126acb-e601-4c95-97cb-4bcf8c179323 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:31.885729 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15126acb-e601-4c95-97cb-4bcf8c179323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:31.906162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-15126acb-e601-4c95-97cb-4bcf8c179323 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:31.907386 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-15126acb-e601-4c95-97cb-4bcf8c179323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:32.016705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c4b30-ea33-483b-9d98-a867eabfee16 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:32.020724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 483/953] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:32.035233 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:32.053016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2828ed-5d9b-4bf2-b650-575e2512ae88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:32.053807 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2828ed-5d9b-4bf2-b650-575e2512ae88 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:32.054384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2828ed-5d9b-4bf2-b650-575e2512ae88 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:32.054758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2828ed-5d9b-4bf2-b650-575e2512ae88 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:32.055560 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2828ed-5d9b-4bf2-b650-575e2512ae88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:32.055974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2828ed-5d9b-4bf2-b650-575e2512ae88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:32.057611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2828ed-5d9b-4bf2-b650-575e2512ae88 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:32.071125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 471/954] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:32.077162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15126acb-e601-4c95-97cb-4bcf8c179323 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:32.177866 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:32.184949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15126acb-e601-4c95-97cb-4bcf8c179323 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:32.221165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c690a-7a9e-4af8-9ba6-c2feec3621a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:32.223004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c690a-7a9e-4af8-9ba6-c2feec3621a6 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:32.223292 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c690a-7a9e-4af8-9ba6-c2feec3621a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:32.224112 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c690a-7a9e-4af8-9ba6-c2feec3621a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:32.231067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c690a-7a9e-4af8-9ba6-c2feec3621a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:32.232503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c690a-7a9e-4af8-9ba6-c2feec3621a6 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-232c690a-7a9e-4af8-9ba6-c2feec3621a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:32.243870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:32.244896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:32.246158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:32.246158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:32.246158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:32.246158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:32.246158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:32.246158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:32.246158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:32.246158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:32.246158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 484/955] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:32.251372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2828ed-5d9b-4bf2-b650-575e2512ae88 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:32.316814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5772-8339-47e6-8a99-bc813ec3dad3 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:32.317140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5772-8339-47e6-8a99-bc813ec3dad3 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:32.317383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5772-8339-47e6-8a99-bc813ec3dad3 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:32.318973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 485/956] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:32.323961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931fb7e-6e94-4130-92d3-ce365d5ba356 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:32.324176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931fb7e-6e94-4130-92d3-ce365d5ba356 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:32.324459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931fb7e-6e94-4130-92d3-ce365d5ba356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:32.342557 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a931fb7e-6e94-4130-92d3-ce365d5ba356 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:32.343754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a931fb7e-6e94-4130-92d3-ce365d5ba356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:32.368465 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2828ed-5d9b-4bf2-b650-575e2512ae88 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:32.404269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857161ab-3eba-45ae-aa92-51efbbd4c4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:32.404916 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857161ab-3eba-45ae-aa92-51efbbd4c4be None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:32.405196 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857161ab-3eba-45ae-aa92-51efbbd4c4be None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:32.405467 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857161ab-3eba-45ae-aa92-51efbbd4c4be None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:32.406150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857161ab-3eba-45ae-aa92-51efbbd4c4be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:32.406461 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857161ab-3eba-45ae-aa92-51efbbd4c4be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:32.407877 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857161ab-3eba-45ae-aa92-51efbbd4c4be None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:32.421339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 472/957] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:32.422453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857161ab-3eba-45ae-aa92-51efbbd4c4be None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:32.428723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 486/958] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:32.428723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931fb7e-6e94-4130-92d3-ce365d5ba356 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:32.467887 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:32.525161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:32.545199 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931fb7e-6e94-4130-92d3-ce365d5ba356 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:32.573786 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857161ab-3eba-45ae-aa92-51efbbd4c4be None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:32.588746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074af612-21bf-48f1-89f4-efc7c711721d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:32.589756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074af612-21bf-48f1-89f4-efc7c711721d None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:32.590374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074af612-21bf-48f1-89f4-efc7c711721d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:32.590983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074af612-21bf-48f1-89f4-efc7c711721d None admin] PATH_INFO: `/v3/projects/fc060764023e44a7ade80eeed98a235f/users/be379e1a6add42b69e9fdd676efca280/roles/f126e5eef4964aa5af83192f79e8a1dd` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:32.600319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074af612-21bf-48f1-89f4-efc7c711721d None admin] RBAC: Authorizing `identity:create_grant(project_id=fc060764023e44a7ade80eeed98a235f, user_id=be379e1a6add42b69e9fdd676efca280, role_id=f126e5eef4964aa5af83192f79e8a1dd)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:32.602243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074af612-21bf-48f1-89f4-efc7c711721d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:32.615144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9aa797-54ce-4479-8792-8f3848a5ce2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:32.615777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aa797-54ce-4479-8792-8f3848a5ce2e None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:32.619723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aa797-54ce-4479-8792-8f3848a5ce2e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:32.619723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aa797-54ce-4479-8792-8f3848a5ce2e None admin] PATH_INFO: `/v3/services/compute` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:32.619723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9aa797-54ce-4479-8792-8f3848a5ce2e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:32.619723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9aa797-54ce-4479-8792-8f3848a5ce2e None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-3b9aa797-54ce-4479-8792-8f3848a5ce2e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:32.627485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:32.629165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:32.630039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:32.630039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:32.630039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:32.630039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:32.630039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:32.630039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:32.630039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:32.630039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:32.630039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:32.630039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:32.630039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:32.630039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:32.630039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:33:32.630039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:32.632952 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 473/959] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:32.634560 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9aa797-54ce-4479-8792-8f3848a5ce2e None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:32.681011 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 487/960] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:33:32 2022] PUT /identity/v3/projects/fc060764023e44a7ade80eeed98a235f/users/be379e1a6add42b69e9fdd676efca280/roles/f126e5eef4964aa5af83192f79e8a1dd => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:33:32.711259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074af612-21bf-48f1-89f4-efc7c711721d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:32.758889 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:32.777303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9aa797-54ce-4479-8792-8f3848a5ce2e None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:32.833590 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87918946-e982-4035-a8f6-5e0190a939e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:32.834486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87918946-e982-4035-a8f6-5e0190a939e4 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:32.834947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87918946-e982-4035-a8f6-5e0190a939e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:32.835370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87918946-e982-4035-a8f6-5e0190a939e4 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:32.836174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87918946-e982-4035-a8f6-5e0190a939e4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:32.836391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074af612-21bf-48f1-89f4-efc7c711721d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:32.836499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87918946-e982-4035-a8f6-5e0190a939e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:32.837925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87918946-e982-4035-a8f6-5e0190a939e4 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:32.844589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 474/961] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 24 07:33:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:32.853276 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87918946-e982-4035-a8f6-5e0190a939e4 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:32.892355 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d12d5-8ed2-4236-ae03-b7b0ccd182ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:32.894418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d12d5-8ed2-4236-ae03-b7b0ccd182ce None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:32.895231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d12d5-8ed2-4236-ae03-b7b0ccd182ce None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:32.895686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d12d5-8ed2-4236-ae03-b7b0ccd182ce None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:32.897083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d12d5-8ed2-4236-ae03-b7b0ccd182ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:32.900361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d12d5-8ed2-4236-ae03-b7b0ccd182ce None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:32.906083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: WARNING keystone.server.flask.common [None req-682d12d5-8ed2-4236-ae03-b7b0ccd182ce None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 07:33:32.975298 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87918946-e982-4035-a8f6-5e0190a939e4 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:33.001707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.sql.core [None req-682d12d5-8ed2-4236-ae03-b7b0ccd182ce None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 07:33:33.001707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 07:33:33.001707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [parameters: {'user_id': '15ca95dca9104f04bfb80d7699270575', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 07:33:33.001707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88249) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-682d12d5-8ed2-4236-ae03-b7b0ccd182ce None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application result.read() Aug 24 07:33:33.019396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:33:33.020621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application yield resource Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise self.value Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application session.commit() Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 07:33:33.021726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self.session.flush() Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application compat.raise_( Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 07:33:33.022635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:33:33.025622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application result.read() Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:33:33.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [parameters: {'user_id': '15ca95dca9104f04bfb80d7699270575', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:33.032417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 07:33:33.038285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 07:33:33.040063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 07:33:33.040063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 07:33:33.040063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 07:33:33.040063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 07:33:33.040063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 07:33:33.040063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:33.040063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 488/962] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Aug 24 07:33:32 2022] POST /identity/v3/users => generated 160 bytes in 310 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:33.040063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d12d5-8ed2-4236-ae03-b7b0ccd182ce None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:33.040063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:33.044628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465e91a-ad08-450b-babb-f44fa3fe837d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:33.045479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465e91a-ad08-450b-babb-f44fa3fe837d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:33.045648 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465e91a-ad08-450b-babb-f44fa3fe837d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:33.048673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465e91a-ad08-450b-babb-f44fa3fe837d None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:33.048673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465e91a-ad08-450b-babb-f44fa3fe837d None admin] RBAC: Adding query filter params (type=compute) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:33.048673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465e91a-ad08-450b-babb-f44fa3fe837d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:33.049779 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465e91a-ad08-450b-babb-f44fa3fe837d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:33.058770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 475/963] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 24 07:33:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:33.160698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d12d5-8ed2-4236-ae03-b7b0ccd182ce None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:33.200909 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de053b-cf22-448a-af28-cbb6ff7fb18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:33.201879 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de053b-cf22-448a-af28-cbb6ff7fb18f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:33.201879 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de053b-cf22-448a-af28-cbb6ff7fb18f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:33.202207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de053b-cf22-448a-af28-cbb6ff7fb18f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:33.213803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de053b-cf22-448a-af28-cbb6ff7fb18f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:33.215721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de053b-cf22-448a-af28-cbb6ff7fb18f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-28de053b-cf22-448a-af28-cbb6ff7fb18f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:33.229010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:33.231238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:33.233509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:33.233509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:33.233509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:33.233509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:33.233509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:33.233509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:33.233509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:33.233509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:33.233509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:33:33.233509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:33.233509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 489/964] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:33:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:33.240689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465e91a-ad08-450b-babb-f44fa3fe837d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:33.259900 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:33.359684 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465e91a-ad08-450b-babb-f44fa3fe837d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:33.398093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fead7-9c6c-4d60-bf44-f0e326c24bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:33.399009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fead7-9c6c-4d60-bf44-f0e326c24bce None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:33.399496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fead7-9c6c-4d60-bf44-f0e326c24bce None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:33.399946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fead7-9c6c-4d60-bf44-f0e326c24bce None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:33.400877 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31fead7-9c6c-4d60-bf44-f0e326c24bce None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:33.401376 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31fead7-9c6c-4d60-bf44-f0e326c24bce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:33.403044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31fead7-9c6c-4d60-bf44-f0e326c24bce None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:33.441089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 476/965] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 24 07:33:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:33.901697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6d394-3b94-43f2-87b7-7c82020f4373 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:33.902121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6d394-3b94-43f2-87b7-7c82020f4373 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:33.902229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6d394-3b94-43f2-87b7-7c82020f4373 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:33.906341 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 490/966] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:33.912037 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2e24-2293-49c5-9e47-d3fdf2283b98 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:33.912522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2e24-2293-49c5-9e47-d3fdf2283b98 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:33.912602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2e24-2293-49c5-9e47-d3fdf2283b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:33.932797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-f7dc2e24-2293-49c5-9e47-d3fdf2283b98 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:33.934052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f7dc2e24-2293-49c5-9e47-d3fdf2283b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:34.005283 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 477/967] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:34.271113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6d394-3b94-43f2-87b7-7c82020f4373 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:34.340231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:34.376248 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6d394-3b94-43f2-87b7-7c82020f4373 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:34.411584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44098e10-db70-4910-9973-b9bdb0ba9bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:34.412200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44098e10-db70-4910-9973-b9bdb0ba9bd3 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:34.412473 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44098e10-db70-4910-9973-b9bdb0ba9bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:34.412767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44098e10-db70-4910-9973-b9bdb0ba9bd3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:34.413340 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44098e10-db70-4910-9973-b9bdb0ba9bd3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:34.414682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44098e10-db70-4910-9973-b9bdb0ba9bd3 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-44098e10-db70-4910-9973-b9bdb0ba9bd3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:34.423148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:34.424124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:34.425546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:34.425546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:34.425546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:34.425546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:34.425546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:34.425546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:34.425546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:34.425546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:34.425546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:34.425546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:34.425546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:34.425546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:34.425546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:34.425546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:34.429010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 491/968] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 24 07:33:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:34.430263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc2e24-2293-49c5-9e47-d3fdf2283b98 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:34.434653 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:34.536565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc2e24-2293-49c5-9e47-d3fdf2283b98 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:34.577496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a7eb6-6ab3-4921-a984-fca9095dc9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:34.578093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a7eb6-6ab3-4921-a984-fca9095dc9b3 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:34.578369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a7eb6-6ab3-4921-a984-fca9095dc9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:34.578643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a7eb6-6ab3-4921-a984-fca9095dc9b3 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:34.579261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a7eb6-6ab3-4921-a984-fca9095dc9b3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:34.579564 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a7eb6-6ab3-4921-a984-fca9095dc9b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:34.580966 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a7eb6-6ab3-4921-a984-fca9095dc9b3 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:34.590705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 478/969] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 24 07:33:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:34.591907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44098e10-db70-4910-9973-b9bdb0ba9bd3 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:34.696466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44098e10-db70-4910-9973-b9bdb0ba9bd3 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:34.725595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:34.731807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96102005-8e58-436c-bf22-ba01f25ed55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:34.732415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96102005-8e58-436c-bf22-ba01f25ed55f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:34.732719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96102005-8e58-436c-bf22-ba01f25ed55f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:34.732991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96102005-8e58-436c-bf22-ba01f25ed55f None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:34.733613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96102005-8e58-436c-bf22-ba01f25ed55f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:34.733910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96102005-8e58-436c-bf22-ba01f25ed55f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:34.735260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96102005-8e58-436c-bf22-ba01f25ed55f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:34.741308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 492/970] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 24 07:33:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:34.747079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a7eb6-6ab3-4921-a984-fca9095dc9b3 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:34.788914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce291b-94a0-4989-8c54-31e46b78384d None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:34.789200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce291b-94a0-4989-8c54-31e46b78384d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:34.789373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce291b-94a0-4989-8c54-31e46b78384d None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:34.790858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 493/971] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:34.796007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdb305-81d6-49ae-ab38-948ce747e489 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:34.796297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdb305-81d6-49ae-ab38-948ce747e489 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:34.796579 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdb305-81d6-49ae-ab38-948ce747e489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:34.815154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-fbfdb305-81d6-49ae-ab38-948ce747e489 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:34.817023 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-fbfdb305-81d6-49ae-ab38-948ce747e489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:34.820308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:34.857799 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a7eb6-6ab3-4921-a984-fca9095dc9b3 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:34.895044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 494/972] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:34.896210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db7862-2b11-4abe-9271-d0f63db06565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:34.897109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db7862-2b11-4abe-9271-d0f63db06565 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:34.897596 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db7862-2b11-4abe-9271-d0f63db06565 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:34.898096 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db7862-2b11-4abe-9271-d0f63db06565 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:34.898999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db7862-2b11-4abe-9271-d0f63db06565 None admin] RBAC: Adding query filter params (interface=admin, service_id=14835d27845f47f3a2ac8bc628094dcc, region_id=RegionOne) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:34.899521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db7862-2b11-4abe-9271-d0f63db06565 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:34.901120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db7862-2b11-4abe-9271-d0f63db06565 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:34.906793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 479/973] 10.222.0.26 () {56 vars in 1249 bytes} [Wed Aug 24 07:33:34 2022] GET /identity/v3/endpoints?service_id=14835d27845f47f3a2ac8bc628094dcc&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:34.912328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdb305-81d6-49ae-ab38-948ce747e489 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:35.013939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdb305-81d6-49ae-ab38-948ce747e489 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:35.037542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:35.048477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459e413-bef1-4912-8c3c-5310a2d1e96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:35.049191 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459e413-bef1-4912-8c3c-5310a2d1e96d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:35.054259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459e413-bef1-4912-8c3c-5310a2d1e96d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:35.054259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459e413-bef1-4912-8c3c-5310a2d1e96d None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:35.054259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459e413-bef1-4912-8c3c-5310a2d1e96d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:35.054259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459e413-bef1-4912-8c3c-5310a2d1e96d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:35.058441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 495/974] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 24 07:33:34 2022] GET /identity/v3/services => generated 1497 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:33:35.117150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065a06f-b906-4f1d-8e8d-da45d39415ed None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:35.117711 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065a06f-b906-4f1d-8e8d-da45d39415ed None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:35.117711 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065a06f-b906-4f1d-8e8d-da45d39415ed None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:35.119306 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 480/975] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:35.124818 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dee16-a986-4d54-84e4-55d97e7fce46 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:35.125108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dee16-a986-4d54-84e4-55d97e7fce46 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:35.125395 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dee16-a986-4d54-84e4-55d97e7fce46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:35.144974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a57dee16-a986-4d54-84e4-55d97e7fce46 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:35.146203 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a57dee16-a986-4d54-84e4-55d97e7fce46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:35.163198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065a06f-b906-4f1d-8e8d-da45d39415ed None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:35.221257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 496/976] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:35.270652 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065a06f-b906-4f1d-8e8d-da45d39415ed None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:35.299086 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:35.305343 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de0dd7-4cb2-40ef-97a7-7218dd13b4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:35.306013 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de0dd7-4cb2-40ef-97a7-7218dd13b4bf None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:35.306336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de0dd7-4cb2-40ef-97a7-7218dd13b4bf None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:35.306664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de0dd7-4cb2-40ef-97a7-7218dd13b4bf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:35.316737 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de0dd7-4cb2-40ef-97a7-7218dd13b4bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-42de0dd7-4cb2-40ef-97a7-7218dd13b4bf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:35.321365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:35.322309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:35.323148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:35.323148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:35.323148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:35.323148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:35.323148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:35.323148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:35.323148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:35.323148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:35.323148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:35.323148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:35.323148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:33:35.323148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:35.323148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 481/977] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Aug 24 07:33:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:35.329742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dee16-a986-4d54-84e4-55d97e7fce46 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:35.429459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dee16-a986-4d54-84e4-55d97e7fce46 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:35.446943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:35.463867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae716ef6-8ed6-4eec-873f-3df8ca77edae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:35.464471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae716ef6-8ed6-4eec-873f-3df8ca77edae None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:35.464778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae716ef6-8ed6-4eec-873f-3df8ca77edae None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:35.465045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae716ef6-8ed6-4eec-873f-3df8ca77edae None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:35.465701 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae716ef6-8ed6-4eec-873f-3df8ca77edae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:35.466036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae716ef6-8ed6-4eec-873f-3df8ca77edae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:35.467386 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae716ef6-8ed6-4eec-873f-3df8ca77edae None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:35.476963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 497/978] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Aug 24 07:33:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:35.482800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de0dd7-4cb2-40ef-97a7-7218dd13b4bf None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:35.490295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae716ef6-8ed6-4eec-873f-3df8ca77edae None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:35.588839 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de0dd7-4cb2-40ef-97a7-7218dd13b4bf None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:35.604467 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae716ef6-8ed6-4eec-873f-3df8ca77edae None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:35.627229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cae908-3b99-48d2-b378-980155954f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:35.628008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cae908-3b99-48d2-b378-980155954f9e None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:35.628284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cae908-3b99-48d2-b378-980155954f9e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:35.628563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cae908-3b99-48d2-b378-980155954f9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:35.638829 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cae908-3b99-48d2-b378-980155954f9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:35.639233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81ac3a-49d7-4236-bb9b-f2143956f21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:35.639835 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81ac3a-49d7-4236-bb9b-f2143956f21c None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:35.640113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81ac3a-49d7-4236-bb9b-f2143956f21c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:35.640384 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81ac3a-49d7-4236-bb9b-f2143956f21c None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:35.640619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cae908-3b99-48d2-b378-980155954f9e None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:35.641058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81ac3a-49d7-4236-bb9b-f2143956f21c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:35.642407 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81ac3a-49d7-4236-bb9b-f2143956f21c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-d5cae908-3b99-48d2-b378-980155954f9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:35.655793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:35.663230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:35.664336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:35.664336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:35.664336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:35.664336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 482/979] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:35.664336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cae908-3b99-48d2-b378-980155954f9e None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:35.709954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:35.727754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 498/980] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 24 07:33:35 2022] POST /identity/v3/services => generated 242 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:33:35.768108 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cae908-3b99-48d2-b378-980155954f9e None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:35.812479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cc8ae-21b4-42ed-abdc-b5d4c683c251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:35.813125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cc8ae-21b4-42ed-abdc-b5d4c683c251 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:35.813404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cc8ae-21b4-42ed-abdc-b5d4c683c251 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:35.813673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cc8ae-21b4-42ed-abdc-b5d4c683c251 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:35.814315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cc8ae-21b4-42ed-abdc-b5d4c683c251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:35.814617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cc8ae-21b4-42ed-abdc-b5d4c683c251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:35.816038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cc8ae-21b4-42ed-abdc-b5d4c683c251 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:35.834020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 483/981] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:35.835652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81ac3a-49d7-4236-bb9b-f2143956f21c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:35.883756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:35.944800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81ac3a-49d7-4236-bb9b-f2143956f21c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:35.980584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ad989-25d5-4d23-bfac-0c2fe96f81e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:35.981243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ad989-25d5-4d23-bfac-0c2fe96f81e1 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:35.981522 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ad989-25d5-4d23-bfac-0c2fe96f81e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:35.981792 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ad989-25d5-4d23-bfac-0c2fe96f81e1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:35.990298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ad989-25d5-4d23-bfac-0c2fe96f81e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:35.991733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ad989-25d5-4d23-bfac-0c2fe96f81e1 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-f59ad989-25d5-4d23-bfac-0c2fe96f81e1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:36.002851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:36.004084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:36.005483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:36.005483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:36.005483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:36.005483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:36.005483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:36.005483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:36.005483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:36.005483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:36.005483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:33:36.005483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:36.006480 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 499/982] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 24 07:33:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:36.009885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cc8ae-21b4-42ed-abdc-b5d4c683c251 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:36.113186 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:36.116424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cc8ae-21b4-42ed-abdc-b5d4c683c251 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:36.149695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcf6a4-62c1-40fa-8342-6a4c2ec8233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:36.150311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcf6a4-62c1-40fa-8342-6a4c2ec8233b None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:36.150534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcf6a4-62c1-40fa-8342-6a4c2ec8233b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:36.150756 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcf6a4-62c1-40fa-8342-6a4c2ec8233b None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:36.151489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcf6a4-62c1-40fa-8342-6a4c2ec8233b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:36.151800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcf6a4-62c1-40fa-8342-6a4c2ec8233b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:36.153295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcf6a4-62c1-40fa-8342-6a4c2ec8233b None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:36.179519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 484/983] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Aug 24 07:33:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:36.180797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ad989-25d5-4d23-bfac-0c2fe96f81e1 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:36.279683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:36.283266 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ad989-25d5-4d23-bfac-0c2fe96f81e1 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:36.318524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27197713-459a-42be-a341-42af1c37c0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:36.319135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27197713-459a-42be-a341-42af1c37c0da None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:36.319421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27197713-459a-42be-a341-42af1c37c0da None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:36.319698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27197713-459a-42be-a341-42af1c37c0da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:36.386573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27197713-459a-42be-a341-42af1c37c0da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:36.388030 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27197713-459a-42be-a341-42af1c37c0da None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:36.482108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:36.491407 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 500/984] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:36 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:33:36.498290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcf6a4-62c1-40fa-8342-6a4c2ec8233b None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:36.605768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcf6a4-62c1-40fa-8342-6a4c2ec8233b None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:36.639979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd018c-c67a-4f74-b5a1-b3618d94b68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:36.640609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd018c-c67a-4f74-b5a1-b3618d94b68d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:36.640919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd018c-c67a-4f74-b5a1-b3618d94b68d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:36.641194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd018c-c67a-4f74-b5a1-b3618d94b68d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:36.651672 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd018c-c67a-4f74-b5a1-b3618d94b68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:36.653143 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd018c-c67a-4f74-b5a1-b3618d94b68d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-3bcd018c-c67a-4f74-b5a1-b3618d94b68d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:36.668101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:36.669282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:36.674790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:36.674790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:36.674790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:36.674790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 485/985] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:36.676485 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27197713-459a-42be-a341-42af1c37c0da None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:36.782208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27197713-459a-42be-a341-42af1c37c0da None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:36.817197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c608dc3-3b7c-4f80-89e4-8de957621136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:36.817805 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c608dc3-3b7c-4f80-89e4-8de957621136 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:36.818084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c608dc3-3b7c-4f80-89e4-8de957621136 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:36.818356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c608dc3-3b7c-4f80-89e4-8de957621136 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:36.818995 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c608dc3-3b7c-4f80-89e4-8de957621136 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:36.819300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c608dc3-3b7c-4f80-89e4-8de957621136 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:36.820685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c608dc3-3b7c-4f80-89e4-8de957621136 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:36.837824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 501/986] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:36.839130 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd018c-c67a-4f74-b5a1-b3618d94b68d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:36.856405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:36.944510 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd018c-c67a-4f74-b5a1-b3618d94b68d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:36.978281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790ca96-4812-4a83-9083-055dea188271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:36.978920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790ca96-4812-4a83-9083-055dea188271 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:36.979205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790ca96-4812-4a83-9083-055dea188271 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:36.979483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790ca96-4812-4a83-9083-055dea188271 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:36.986184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790ca96-4812-4a83-9083-055dea188271 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:36.988367 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790ca96-4812-4a83-9083-055dea188271 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-d790ca96-4812-4a83-9083-055dea188271 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:36.999643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:37.000532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:37.002071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:37.002071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:37.002071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:37.002071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:37.002071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:37.002071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:37.002071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:37.002071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:37.002071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 486/987] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:37.006589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c608dc3-3b7c-4f80-89e4-8de957621136 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:37.010936 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c2c91-0989-4d5a-9f32-5ae1d2f4a226 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:37.011222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c2c91-0989-4d5a-9f32-5ae1d2f4a226 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:37.011491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c2c91-0989-4d5a-9f32-5ae1d2f4a226 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:37.012884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 487/988] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:37.018930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c401461-d0e5-4a07-ad4d-950a0451b98d None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:37.018930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c401461-d0e5-4a07-ad4d-950a0451b98d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:37.018930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c401461-d0e5-4a07-ad4d-950a0451b98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:37.038935 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-6c401461-d0e5-4a07-ad4d-950a0451b98d None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:37.040045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6c401461-d0e5-4a07-ad4d-950a0451b98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:37.074534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:37.105977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:37.111754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 488/989] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:37.117754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c608dc3-3b7c-4f80-89e4-8de957621136 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:37.152815 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10d858-eb4f-4cdf-8712-71ad71c1833f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:37.153699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10d858-eb4f-4cdf-8712-71ad71c1833f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:37.154304 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10d858-eb4f-4cdf-8712-71ad71c1833f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:37.154759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10d858-eb4f-4cdf-8712-71ad71c1833f None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:37.155664 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10d858-eb4f-4cdf-8712-71ad71c1833f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:37.156176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10d858-eb4f-4cdf-8712-71ad71c1833f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:37.157806 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10d858-eb4f-4cdf-8712-71ad71c1833f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:37.175427 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 502/990] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:37.176794 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c401461-d0e5-4a07-ad4d-950a0451b98d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:37.281543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c401461-d0e5-4a07-ad4d-950a0451b98d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:37.317133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaba15-a9f9-4ce2-9dc0-22732e0f1bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:37.317734 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaba15-a9f9-4ce2-9dc0-22732e0f1bcf None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:37.318012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaba15-a9f9-4ce2-9dc0-22732e0f1bcf None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:37.318283 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaba15-a9f9-4ce2-9dc0-22732e0f1bcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:37.318995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eaba15-a9f9-4ce2-9dc0-22732e0f1bcf None admin] RBAC: Adding query filter params (role.id=f126e5eef4964aa5af83192f79e8a1dd, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=be379e1a6add42b69e9fdd676efca280) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:37.319301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eaba15-a9f9-4ce2-9dc0-22732e0f1bcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:37.320742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eaba15-a9f9-4ce2-9dc0-22732e0f1bcf None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:37.326724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 489/991] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:37 2022] GET /identity/v3/role_assignments?user.id=be379e1a6add42b69e9fdd676efca280&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=f126e5eef4964aa5af83192f79e8a1dd => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:37.382952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10d858-eb4f-4cdf-8712-71ad71c1833f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:37.492490 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10d858-eb4f-4cdf-8712-71ad71c1833f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:37.514558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:37.535740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06852dc-a81b-4a9e-b113-bd7323a4758e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:37.536611 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06852dc-a81b-4a9e-b113-bd7323a4758e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:37.536983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06852dc-a81b-4a9e-b113-bd7323a4758e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:37.537658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06852dc-a81b-4a9e-b113-bd7323a4758e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:37.537859 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06852dc-a81b-4a9e-b113-bd7323a4758e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:37.539218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06852dc-a81b-4a9e-b113-bd7323a4758e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-c06852dc-a81b-4a9e-b113-bd7323a4758e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:37.549282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:37.550907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:37.551751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 503/992] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 24 07:33:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:37.558717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaba15-a9f9-4ce2-9dc0-22732e0f1bcf None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:37.625108 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:37.664858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaba15-a9f9-4ce2-9dc0-22732e0f1bcf None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:37.683361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2d29e-1192-49f7-bde4-6556ed50a59c None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:37.683648 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2d29e-1192-49f7-bde4-6556ed50a59c None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:37.683930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2d29e-1192-49f7-bde4-6556ed50a59c None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:37.685464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 504/993] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:37.690876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755206a0-4285-414e-bd6b-87e37d8c86c7 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:37.691481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755206a0-4285-414e-bd6b-87e37d8c86c7 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:37.691768 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755206a0-4285-414e-bd6b-87e37d8c86c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:37.700342 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cc49a-cbc6-4dc3-b0fe-2435ca8fa39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:37.700983 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cc49a-cbc6-4dc3-b0fe-2435ca8fa39e None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:37.701269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cc49a-cbc6-4dc3-b0fe-2435ca8fa39e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:37.701546 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cc49a-cbc6-4dc3-b0fe-2435ca8fa39e None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:37.702175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cc49a-cbc6-4dc3-b0fe-2435ca8fa39e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:37.702481 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cc49a-cbc6-4dc3-b0fe-2435ca8fa39e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:37.703844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cc49a-cbc6-4dc3-b0fe-2435ca8fa39e None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:37.709570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 490/994] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 24 07:33:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:37.712690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-755206a0-4285-414e-bd6b-87e37d8c86c7 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:37.712690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-755206a0-4285-414e-bd6b-87e37d8c86c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:37.715321 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cc49a-cbc6-4dc3-b0fe-2435ca8fa39e None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:37.787464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 505/995] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:37.819929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cc49a-cbc6-4dc3-b0fe-2435ca8fa39e None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:37.848300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:37.856016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0ffed-00bd-4b7f-bfab-1cba9df889ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:37.856618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0ffed-00bd-4b7f-bfab-1cba9df889ab None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:37.856923 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0ffed-00bd-4b7f-bfab-1cba9df889ab None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:37.857200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0ffed-00bd-4b7f-bfab-1cba9df889ab None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:37.857826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0ffed-00bd-4b7f-bfab-1cba9df889ab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:37.858133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0ffed-00bd-4b7f-bfab-1cba9df889ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:37.859499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0ffed-00bd-4b7f-bfab-1cba9df889ab None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:37.865136 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 491/996] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 24 07:33:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:37.870093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755206a0-4285-414e-bd6b-87e37d8c86c7 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:37.972553 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755206a0-4285-414e-bd6b-87e37d8c86c7 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:37.986624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:38.006721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441f62d-cc80-441a-874b-e3b929753e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:38.007329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441f62d-cc80-441a-874b-e3b929753e72 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:38.007612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441f62d-cc80-441a-874b-e3b929753e72 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:38.007888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441f62d-cc80-441a-874b-e3b929753e72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:38.008586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441f62d-cc80-441a-874b-e3b929753e72 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:38.009984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441f62d-cc80-441a-874b-e3b929753e72 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:38.064189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0ffed-00bd-4b7f-bfab-1cba9df889ab None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:38.120852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 506/997] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:33:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:33:38.183335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0ffed-00bd-4b7f-bfab-1cba9df889ab None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:38.227577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd6e79-b298-4ecd-a576-b3a1c29914ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:38.228137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6e79-b298-4ecd-a576-b3a1c29914ee None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:38.228411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6e79-b298-4ecd-a576-b3a1c29914ee None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:38.228702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6e79-b298-4ecd-a576-b3a1c29914ee None admin] PATH_INFO: `/v3/services/compute` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:38.229294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd6e79-b298-4ecd-a576-b3a1c29914ee None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:38.230672 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd6e79-b298-4ecd-a576-b3a1c29914ee None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-81cd6e79-b298-4ecd-a576-b3a1c29914ee None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:38.240522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:38.241708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:38.242607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 492/998] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:38.249945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441f62d-cc80-441a-874b-e3b929753e72 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:38.357586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441f62d-cc80-441a-874b-e3b929753e72 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:38.391868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ed501-8e9d-4afa-bcb1-dbad92c78ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:38.392473 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ed501-8e9d-4afa-bcb1-dbad92c78ffd None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:38.392783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ed501-8e9d-4afa-bcb1-dbad92c78ffd None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:38.393057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ed501-8e9d-4afa-bcb1-dbad92c78ffd None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:38.393969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ed501-8e9d-4afa-bcb1-dbad92c78ffd None admin] RBAC: Adding query filter params (name=compute) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:38.394279 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ed501-8e9d-4afa-bcb1-dbad92c78ffd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:38.395648 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ed501-8e9d-4afa-bcb1-dbad92c78ffd None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:38.400755 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 507/999] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 24 07:33:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:38.406086 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd6e79-b298-4ecd-a576-b3a1c29914ee None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:38.461078 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:38.511091 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd6e79-b298-4ecd-a576-b3a1c29914ee None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:38.552193 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92c9cb-a100-499c-ac15-ac5884097e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:38.552829 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92c9cb-a100-499c-ac15-ac5884097e88 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:38.553114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92c9cb-a100-499c-ac15-ac5884097e88 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:38.553388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92c9cb-a100-499c-ac15-ac5884097e88 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:38.554040 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92c9cb-a100-499c-ac15-ac5884097e88 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:38.554403 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92c9cb-a100-499c-ac15-ac5884097e88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:38.555785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92c9cb-a100-499c-ac15-ac5884097e88 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:38.565446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 493/1000] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 24 07:33:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:38.566947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ed501-8e9d-4afa-bcb1-dbad92c78ffd None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:38.614410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:38.672892 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ed501-8e9d-4afa-bcb1-dbad92c78ffd None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:38.707934 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969003d-c9dc-426c-ad38-9aa6562c6c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:38.708541 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969003d-c9dc-426c-ad38-9aa6562c6c48 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:38.708850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969003d-c9dc-426c-ad38-9aa6562c6c48 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:38.709125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969003d-c9dc-426c-ad38-9aa6562c6c48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:38.709822 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969003d-c9dc-426c-ad38-9aa6562c6c48 None admin] RBAC: Adding query filter params (interface=public, service_id=252cca2563c04db3a71c246617160a20, region_id=RegionOne) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:38.710193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969003d-c9dc-426c-ad38-9aa6562c6c48 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:38.711560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969003d-c9dc-426c-ad38-9aa6562c6c48 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:38.721240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 508/1001] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 24 07:33:38 2022] GET /identity/v3/endpoints?service_id=252cca2563c04db3a71c246617160a20&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:38.722681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92c9cb-a100-499c-ac15-ac5884097e88 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:38.824789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92c9cb-a100-499c-ac15-ac5884097e88 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:38.841563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:38.859639 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b7d09-d7a3-4720-bacd-83ccace028ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:38.860246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b7d09-d7a3-4720-bacd-83ccace028ec None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:38.860526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b7d09-d7a3-4720-bacd-83ccace028ec None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:38.860829 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b7d09-d7a3-4720-bacd-83ccace028ec None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:38.861493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b7d09-d7a3-4720-bacd-83ccace028ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:38.863760 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b7d09-d7a3-4720-bacd-83ccace028ec None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:38.876671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 494/1002] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 24 07:33:38 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:33:39.315142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f75b69-ab6d-4afd-9bdd-df95c0106c88 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:39.315571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f75b69-ab6d-4afd-9bdd-df95c0106c88 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:39.315862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f75b69-ab6d-4afd-9bdd-df95c0106c88 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:39.317253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 509/1003] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:39.322541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d44aca-2cf6-41f6-820a-7c4aa5604718 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:39.322974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d44aca-2cf6-41f6-820a-7c4aa5604718 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:39.323155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d44aca-2cf6-41f6-820a-7c4aa5604718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:39.341531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d6d44aca-2cf6-41f6-820a-7c4aa5604718 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:39.342719 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d6d44aca-2cf6-41f6-820a-7c4aa5604718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:39.412317 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 495/1004] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:39.680500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f75b69-ab6d-4afd-9bdd-df95c0106c88 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:39.786815 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f75b69-ab6d-4afd-9bdd-df95c0106c88 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:39.788338 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:39.820909 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692e32b-cf87-4c0f-9f85-152ad6818b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:39.821567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692e32b-cf87-4c0f-9f85-152ad6818b52 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:39.821838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692e32b-cf87-4c0f-9f85-152ad6818b52 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:39.822497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692e32b-cf87-4c0f-9f85-152ad6818b52 None admin] PATH_INFO: `/v3/services/image` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:39.823318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692e32b-cf87-4c0f-9f85-152ad6818b52 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:39.825157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692e32b-cf87-4c0f-9f85-152ad6818b52 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-f692e32b-cf87-4c0f-9f85-152ad6818b52 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:39.834571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:39.835583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:39.836424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:39.836424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:39.836424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:39.836424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:39.836424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:39.836424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:39.836424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:39.836424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:39.836424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:39.836424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:39.836424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:39.836424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:39.836424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:33:39.836424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:39.840616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 510/1005] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Aug 24 07:33:39 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:39.841858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d44aca-2cf6-41f6-820a-7c4aa5604718 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:39.944921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d44aca-2cf6-41f6-820a-7c4aa5604718 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:39.957119 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:39.982521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2e899-3668-4c09-982e-11cc513f782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:39.983135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2e899-3668-4c09-982e-11cc513f782c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:39.983414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2e899-3668-4c09-982e-11cc513f782c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:39.983685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2e899-3668-4c09-982e-11cc513f782c None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:39.984311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2e899-3668-4c09-982e-11cc513f782c None admin] RBAC: Adding query filter params (name=image) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:39.984617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2e899-3668-4c09-982e-11cc513f782c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:39.986064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2e899-3668-4c09-982e-11cc513f782c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:39.996705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 496/1006] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:39.998346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692e32b-cf87-4c0f-9f85-152ad6818b52 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:40.051214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404d7d7-f0bd-4b5e-92ab-6a68f997e677 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:40.051494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404d7d7-f0bd-4b5e-92ab-6a68f997e677 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:40.051763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404d7d7-f0bd-4b5e-92ab-6a68f997e677 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:40.053277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 497/1007] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:40.057927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddaeee-4e8d-4013-bd7f-f567cdc90ddd None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:40.058205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddaeee-4e8d-4013-bd7f-f567cdc90ddd None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:40.058480 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddaeee-4e8d-4013-bd7f-f567cdc90ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:40.078054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e3ddaeee-4e8d-4013-bd7f-f567cdc90ddd None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:40.079633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e3ddaeee-4e8d-4013-bd7f-f567cdc90ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:40.130520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692e32b-cf87-4c0f-9f85-152ad6818b52 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:40.165044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 498/1008] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:40.167161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102bdba-68b8-4f87-bbfe-ed719e86bc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:40.167838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102bdba-68b8-4f87-bbfe-ed719e86bc05 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:40.168195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102bdba-68b8-4f87-bbfe-ed719e86bc05 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:40.168542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102bdba-68b8-4f87-bbfe-ed719e86bc05 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:40.169269 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102bdba-68b8-4f87-bbfe-ed719e86bc05 None admin] RBAC: Adding query filter params (type=image) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:40.169643 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102bdba-68b8-4f87-bbfe-ed719e86bc05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:40.171096 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102bdba-68b8-4f87-bbfe-ed719e86bc05 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:40.175860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 511/1009] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:39 2022] GET /identity/v3/services?type=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:40.425965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddaeee-4e8d-4013-bd7f-f567cdc90ddd None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:40.524666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddaeee-4e8d-4013-bd7f-f567cdc90ddd None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:40.529960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:40.563197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83a44c-e833-4919-96a5-a773d782bb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:40.563833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83a44c-e833-4919-96a5-a773d782bb37 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:40.564115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83a44c-e833-4919-96a5-a773d782bb37 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:40.564388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83a44c-e833-4919-96a5-a773d782bb37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:40.565695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83a44c-e833-4919-96a5-a773d782bb37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:40.567196 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83a44c-e833-4919-96a5-a773d782bb37 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:40.568959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 499/1010] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:40.574531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102bdba-68b8-4f87-bbfe-ed719e86bc05 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:40.610736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:40.684990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102bdba-68b8-4f87-bbfe-ed719e86bc05 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:40.719791 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467341c-912e-4d6a-9327-4a1a5ccc495d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:40.720402 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467341c-912e-4d6a-9327-4a1a5ccc495d None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:40.720709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467341c-912e-4d6a-9327-4a1a5ccc495d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:40.720988 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467341c-912e-4d6a-9327-4a1a5ccc495d None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:40.722441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467341c-912e-4d6a-9327-4a1a5ccc495d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:40.723871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467341c-912e-4d6a-9327-4a1a5ccc495d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:40.809033 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c387c-7bfa-4d4f-8896-69ce25c665d5 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:40.809454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c387c-7bfa-4d4f-8896-69ce25c665d5 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:40.809610 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c387c-7bfa-4d4f-8896-69ce25c665d5 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:40.811033 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 500/1011] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:40.820821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 512/1012] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 24 07:33:40 2022] POST /identity/v3/projects => generated 311 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:33:40.822602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5496f8-2e1c-499a-b3d1-7667c71b210e None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:40.822899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5496f8-2e1c-499a-b3d1-7667c71b210e None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:40.823167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5496f8-2e1c-499a-b3d1-7667c71b210e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:40.842375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-9f5496f8-2e1c-499a-b3d1-7667c71b210e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:40.843612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9f5496f8-2e1c-499a-b3d1-7667c71b210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:40.918476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 501/1013] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:41.187495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467341c-912e-4d6a-9327-4a1a5ccc495d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:41.274610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:41.293306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467341c-912e-4d6a-9327-4a1a5ccc495d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:41.329620 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5c109-5edb-4021-8a4d-71668292c6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:41.330261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5c109-5edb-4021-8a4d-71668292c6a3 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:41.330545 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5c109-5edb-4021-8a4d-71668292c6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:41.330856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5c109-5edb-4021-8a4d-71668292c6a3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:41.331595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5c109-5edb-4021-8a4d-71668292c6a3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:41.333005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5c109-5edb-4021-8a4d-71668292c6a3 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-5ec5c109-5edb-4021-8a4d-71668292c6a3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:41.341761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:41.342750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:41.343641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 513/1014] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:41.349587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5496f8-2e1c-499a-b3d1-7667c71b210e None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:41.448066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5496f8-2e1c-499a-b3d1-7667c71b210e None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:41.449508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:41.483585 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d441c76-092f-4703-a5c1-f127bb10a167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:41.484189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d441c76-092f-4703-a5c1-f127bb10a167 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:41.484470 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d441c76-092f-4703-a5c1-f127bb10a167 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:41.484844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d441c76-092f-4703-a5c1-f127bb10a167 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:41.485478 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d441c76-092f-4703-a5c1-f127bb10a167 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:41.485789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d441c76-092f-4703-a5c1-f127bb10a167 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:41.487253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d441c76-092f-4703-a5c1-f127bb10a167 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:41.496792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 502/1015] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 24 07:33:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:41.498335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5c109-5edb-4021-8a4d-71668292c6a3 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:41.607822 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5c109-5edb-4021-8a4d-71668292c6a3 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:41.642518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c86581-a4cd-43e0-b201-b333b74cd455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:41.643115 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c86581-a4cd-43e0-b201-b333b74cd455 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:41.643391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c86581-a4cd-43e0-b201-b333b74cd455 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:41.643664 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c86581-a4cd-43e0-b201-b333b74cd455 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:41.644279 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c86581-a4cd-43e0-b201-b333b74cd455 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:41.644576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c86581-a4cd-43e0-b201-b333b74cd455 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:41.645964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c86581-a4cd-43e0-b201-b333b74cd455 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:41.650921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 514/1016] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 24 07:33:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:41.656683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d441c76-092f-4703-a5c1-f127bb10a167 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:41.757569 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d441c76-092f-4703-a5c1-f127bb10a167 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:41.792216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b4c45-46fb-489d-aa32-bfea21b81435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:41.792853 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b4c45-46fb-489d-aa32-bfea21b81435 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:41.793130 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b4c45-46fb-489d-aa32-bfea21b81435 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:41.793397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b4c45-46fb-489d-aa32-bfea21b81435 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:41.794060 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b4c45-46fb-489d-aa32-bfea21b81435 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:41.795413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b4c45-46fb-489d-aa32-bfea21b81435 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:41.869197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 503/1017] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:33:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:33:42.116053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace354e6-2aea-400c-9ff2-48c96fb37fb7 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:42.117040 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace354e6-2aea-400c-9ff2-48c96fb37fb7 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:42.117573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace354e6-2aea-400c-9ff2-48c96fb37fb7 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:42.118916 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 515/1018] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:42.124486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c51a4-c66d-45b1-a87a-a1ea7b2b47ca None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:42.124812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c51a4-c66d-45b1-a87a-a1ea7b2b47ca None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:42.125093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c51a4-c66d-45b1-a87a-a1ea7b2b47ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:42.145077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-fa0c51a4-c66d-45b1-a87a-a1ea7b2b47ca None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:42.146269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fa0c51a4-c66d-45b1-a87a-a1ea7b2b47ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:42.164015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:42.222821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 504/1019] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:42 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:42.495427 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace354e6-2aea-400c-9ff2-48c96fb37fb7 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:42.508615 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:42.605785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace354e6-2aea-400c-9ff2-48c96fb37fb7 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:42.641006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb5b26-8182-4075-a5f4-0f7a4cf99b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:42.641624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb5b26-8182-4075-a5f4-0f7a4cf99b81 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:42.641896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb5b26-8182-4075-a5f4-0f7a4cf99b81 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:42.642168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb5b26-8182-4075-a5f4-0f7a4cf99b81 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:42.642823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb5b26-8182-4075-a5f4-0f7a4cf99b81 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:42.644895 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb5b26-8182-4075-a5f4-0f7a4cf99b81 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:42.710730 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 516/1020] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 24 07:33:42 2022] POST /identity/v3/services => generated 242 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:33:42.792336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbac480-0e89-4d04-9642-3d1e878b8935 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:42.792897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbac480-0e89-4d04-9642-3d1e878b8935 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:42.792897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbac480-0e89-4d04-9642-3d1e878b8935 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:42.794343 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 505/1021] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:42.799887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7bf1e-21c6-4ead-8afa-9e23149ab9db None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:42.800188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7bf1e-21c6-4ead-8afa-9e23149ab9db None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:42.800450 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7bf1e-21c6-4ead-8afa-9e23149ab9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:42.820164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0fe7bf1e-21c6-4ead-8afa-9e23149ab9db None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:42.821424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0fe7bf1e-21c6-4ead-8afa-9e23149ab9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:42.891099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:42.902218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 517/1022] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:43.169011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbac480-0e89-4d04-9642-3d1e878b8935 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:43.272592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbac480-0e89-4d04-9642-3d1e878b8935 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:43.306305 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17bd17-0428-43a3-a7ac-479ae9d35be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:43.306932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17bd17-0428-43a3-a7ac-479ae9d35be3 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:43.307212 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17bd17-0428-43a3-a7ac-479ae9d35be3 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:43.307483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17bd17-0428-43a3-a7ac-479ae9d35be3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:43.308592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17bd17-0428-43a3-a7ac-479ae9d35be3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:43.310046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17bd17-0428-43a3-a7ac-479ae9d35be3 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:43.311768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 506/1023] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:43.317429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7bf1e-21c6-4ead-8afa-9e23149ab9db None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:43.415811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7bf1e-21c6-4ead-8afa-9e23149ab9db None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:43.449448 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7a429-2b7a-44eb-93f7-9af47fac5f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:43.449983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7a429-2b7a-44eb-93f7-9af47fac5f66 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:43.450262 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7a429-2b7a-44eb-93f7-9af47fac5f66 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:43.450533 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7a429-2b7a-44eb-93f7-9af47fac5f66 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:43.451182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7a429-2b7a-44eb-93f7-9af47fac5f66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:43.452745 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7a429-2b7a-44eb-93f7-9af47fac5f66 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:43.584862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 518/1024] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:33:43 2022] POST /identity/v3/users => generated 285 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:33:43.886268 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3388328-2c27-46fd-be97-077decfa9d57 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:43.886793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3388328-2c27-46fd-be97-077decfa9d57 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:43.886793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3388328-2c27-46fd-be97-077decfa9d57 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:43.888268 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 507/1025] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:43.893913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076d38c-8b9e-4061-862f-371bbff8ae64 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:43.894424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076d38c-8b9e-4061-862f-371bbff8ae64 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:43.894721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076d38c-8b9e-4061-862f-371bbff8ae64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:43.912601 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0076d38c-8b9e-4061-862f-371bbff8ae64 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:43.913810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0076d38c-8b9e-4061-862f-371bbff8ae64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:43.962253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:43.981514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 519/1026] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:44.248695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3388328-2c27-46fd-be97-077decfa9d57 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:44.267538 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:44.352677 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3388328-2c27-46fd-be97-077decfa9d57 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:44.386093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8968d56-7130-49fe-b657-607eb9d1f35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:44.386721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8968d56-7130-49fe-b657-607eb9d1f35b None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:44.386998 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8968d56-7130-49fe-b657-607eb9d1f35b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:44.387266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8968d56-7130-49fe-b657-607eb9d1f35b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:44.398282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8968d56-7130-49fe-b657-607eb9d1f35b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-f8968d56-7130-49fe-b657-607eb9d1f35b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:44.402930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:44.403953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:44.404882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:44.404882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:44.404882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:44.404882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:44.404882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:44.404882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:44.404882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:44.404882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:44.404882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:44.404882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:44.404882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:33:44.404882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:44.408717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 508/1027] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Aug 24 07:33:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:44.410363 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076d38c-8b9e-4061-862f-371bbff8ae64 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:44.508029 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076d38c-8b9e-4061-862f-371bbff8ae64 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:44.546054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91bbea-c657-41f6-9cb2-0427b4e92f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:44.546659 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91bbea-c657-41f6-9cb2-0427b4e92f71 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:44.546935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91bbea-c657-41f6-9cb2-0427b4e92f71 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:44.547203 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91bbea-c657-41f6-9cb2-0427b4e92f71 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:44.547862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91bbea-c657-41f6-9cb2-0427b4e92f71 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:44.548162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91bbea-c657-41f6-9cb2-0427b4e92f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:44.550076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91bbea-c657-41f6-9cb2-0427b4e92f71 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:44.562800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 520/1028] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Aug 24 07:33:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:44.564169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8968d56-7130-49fe-b657-607eb9d1f35b None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:44.633952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0517c-2875-4b77-861e-1a9f25e1da37 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:44.634283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0517c-2875-4b77-861e-1a9f25e1da37 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:44.634515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0517c-2875-4b77-861e-1a9f25e1da37 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:44.635853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 521/1029] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:44.640713 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2f437-7566-41c2-989a-c7d14021df69 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:44.640967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2f437-7566-41c2-989a-c7d14021df69 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:44.641249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2f437-7566-41c2-989a-c7d14021df69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:44.653932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:44.660443 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-b0d2f437-7566-41c2-989a-c7d14021df69 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:44.661678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b0d2f437-7566-41c2-989a-c7d14021df69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:44.683562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8968d56-7130-49fe-b657-607eb9d1f35b None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:44.704118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:44.724442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdada822-a26d-4b47-a4d3-c20223a7faa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:44.725105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdada822-a26d-4b47-a4d3-c20223a7faa1 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:44.725383 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdada822-a26d-4b47-a4d3-c20223a7faa1 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:44.725658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdada822-a26d-4b47-a4d3-c20223a7faa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:44.733688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 522/1030] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:44.740767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdada822-a26d-4b47-a4d3-c20223a7faa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:44.742364 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdada822-a26d-4b47-a4d3-c20223a7faa1 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-cdada822-a26d-4b47-a4d3-c20223a7faa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:44.758540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:44.759499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:44.765673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:44.765673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:44.765673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:44.765673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 509/1031] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:44.765910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2f437-7566-41c2-989a-c7d14021df69 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:44.864414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2f437-7566-41c2-989a-c7d14021df69 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:44.898298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c386a-4538-4471-a025-62ed9fb01369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:44.899200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c386a-4538-4471-a025-62ed9fb01369 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:44.899636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c386a-4538-4471-a025-62ed9fb01369 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:44.900056 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c386a-4538-4471-a025-62ed9fb01369 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:44.900881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c386a-4538-4471-a025-62ed9fb01369 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:44.901348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c386a-4538-4471-a025-62ed9fb01369 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:44.902857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c386a-4538-4471-a025-62ed9fb01369 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:44.918507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 523/1032] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:44.919693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdada822-a26d-4b47-a4d3-c20223a7faa1 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:45.001761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c386a-4538-4471-a025-62ed9fb01369 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:45.027158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdada822-a26d-4b47-a4d3-c20223a7faa1 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:45.041579 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:45.055752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:45.063421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee913adf-3391-44bd-9222-9f20e3a5cf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:45.064047 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee913adf-3391-44bd-9222-9f20e3a5cf7a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:45.064338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee913adf-3391-44bd-9222-9f20e3a5cf7a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:45.064608 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee913adf-3391-44bd-9222-9f20e3a5cf7a None admin] PATH_INFO: `/v3/users/nova` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:45.072834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee913adf-3391-44bd-9222-9f20e3a5cf7a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:45.080947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee913adf-3391-44bd-9222-9f20e3a5cf7a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-ee913adf-3391-44bd-9222-9f20e3a5cf7a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:45.085202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:45.086157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:45.087421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:45.087421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:45.087421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:45.087421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:45.087421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:45.087421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:45.087421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:45.087421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:45.087421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:33:45.087421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:45.087421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 510/1033] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 24 07:33:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:45.091939 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee913adf-3391-44bd-9222-9f20e3a5cf7a None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:45.118659 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c386a-4538-4471-a025-62ed9fb01369 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:45.156807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ceaea-b2ea-4655-812a-634c805c131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:45.157590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ceaea-b2ea-4655-812a-634c805c131e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:45.157933 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ceaea-b2ea-4655-812a-634c805c131e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:45.158275 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ceaea-b2ea-4655-812a-634c805c131e None admin] PATH_INFO: `/v3/services/image` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:45.159456 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ceaea-b2ea-4655-812a-634c805c131e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:45.160926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ceaea-b2ea-4655-812a-634c805c131e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-8e3ceaea-b2ea-4655-812a-634c805c131e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:45.169785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:45.170996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:45.172293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:45.172293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:45.172293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:45.172293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:45.172293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:45.172293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:45.172293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:45.172293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:45.172293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:45.172293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:45.172293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:45.172293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:45.172293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:33:45.172293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:45.176176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 524/1034] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Aug 24 07:33:45 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:45.177271 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ceaea-b2ea-4655-812a-634c805c131e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:45.205008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee913adf-3391-44bd-9222-9f20e3a5cf7a None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:45.242116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dbe57-4a03-46d0-814d-12250f478ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:45.242865 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dbe57-4a03-46d0-814d-12250f478ac1 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:45.243255 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dbe57-4a03-46d0-814d-12250f478ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:45.243641 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dbe57-4a03-46d0-814d-12250f478ac1 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:45.244463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dbe57-4a03-46d0-814d-12250f478ac1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:45.244927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dbe57-4a03-46d0-814d-12250f478ac1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:45.246450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dbe57-4a03-46d0-814d-12250f478ac1 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:45.265396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 511/1035] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 24 07:33:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:45.271858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dbe57-4a03-46d0-814d-12250f478ac1 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:45.281171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:45.285491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ceaea-b2ea-4655-812a-634c805c131e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:45.321658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68053103-6334-4661-abb2-ae23bd3bc15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:45.322361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68053103-6334-4661-abb2-ae23bd3bc15a None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:45.322664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:45.322888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68053103-6334-4661-abb2-ae23bd3bc15a None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:45.323093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68053103-6334-4661-abb2-ae23bd3bc15a None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:45.323795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68053103-6334-4661-abb2-ae23bd3bc15a None admin] RBAC: Adding query filter params (name=image) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:45.324158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68053103-6334-4661-abb2-ae23bd3bc15a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:45.325620 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68053103-6334-4661-abb2-ae23bd3bc15a None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:45.331310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 525/1036] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:45.337578 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68053103-6334-4661-abb2-ae23bd3bc15a None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:45.380689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dbe57-4a03-46d0-814d-12250f478ac1 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:45.414998 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5115c-3db5-4571-a720-75d0078d4d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:45.415600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5115c-3db5-4571-a720-75d0078d4d81 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:45.415882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5115c-3db5-4571-a720-75d0078d4d81 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:45.416155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5115c-3db5-4571-a720-75d0078d4d81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:45.445719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68053103-6334-4661-abb2-ae23bd3bc15a None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:45.482909 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8d61a-dadd-46cb-9f80-8147a4b26344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:45.483544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8d61a-dadd-46cb-9f80-8147a4b26344 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:45.483825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8d61a-dadd-46cb-9f80-8147a4b26344 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:45.484099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8d61a-dadd-46cb-9f80-8147a4b26344 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:45.484753 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8d61a-dadd-46cb-9f80-8147a4b26344 None admin] RBAC: Adding query filter params (type=image) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:45.485062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8d61a-dadd-46cb-9f80-8147a4b26344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:45.486427 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8d61a-dadd-46cb-9f80-8147a4b26344 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:45.490594 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5115c-3db5-4571-a720-75d0078d4d81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:45.492686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 526/1037] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:45.492809 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5115c-3db5-4571-a720-75d0078d4d81 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:45.497712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8d61a-dadd-46cb-9f80-8147a4b26344 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:45.530267 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:45.545897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:45.600902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8d61a-dadd-46cb-9f80-8147a4b26344 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:45.601439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 512/1038] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:45 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:33:45.602689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6ead3-115d-4fb1-98f9-d3cef809e0ab None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:45.602971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6ead3-115d-4fb1-98f9-d3cef809e0ab None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:45.603919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6ead3-115d-4fb1-98f9-d3cef809e0ab None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:45.608437 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 513/1039] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:45.609724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7ad7b-ba89-4978-8c2e-1fbd27ae3130 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:45.610004 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7ad7b-ba89-4978-8c2e-1fbd27ae3130 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:45.610271 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7ad7b-ba89-4978-8c2e-1fbd27ae3130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:45.629198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-72c7ad7b-ba89-4978-8c2e-1fbd27ae3130 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:45.630426 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-72c7ad7b-ba89-4978-8c2e-1fbd27ae3130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:45.639101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c7276-b6ba-4d89-97cb-0a623da9e568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:45.639710 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c7276-b6ba-4d89-97cb-0a623da9e568 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:45.639991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c7276-b6ba-4d89-97cb-0a623da9e568 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:45.640265 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c7276-b6ba-4d89-97cb-0a623da9e568 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:45.641000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c7276-b6ba-4d89-97cb-0a623da9e568 None admin] RBAC: Adding query filter params (interface=public, service_id=467d11ee07314aaeadf5f59560e3d482, region_id=RegionOne) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:45.641310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c7276-b6ba-4d89-97cb-0a623da9e568 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:45.642671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c7276-b6ba-4d89-97cb-0a623da9e568 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:45.647737 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 527/1040] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 24 07:33:45 2022] GET /identity/v3/endpoints?service_id=467d11ee07314aaeadf5f59560e3d482&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:45.648352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c7276-b6ba-4d89-97cb-0a623da9e568 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:45.695858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 514/1041] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:45.697576 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7ad7b-ba89-4978-8c2e-1fbd27ae3130 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:45.741635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:45.751637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c7276-b6ba-4d89-97cb-0a623da9e568 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:45.776725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:45.788338 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e60bc2-2a1f-4db8-9ef2-edbef1ddcac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:45.788955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e60bc2-2a1f-4db8-9ef2-edbef1ddcac9 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:45.789239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e60bc2-2a1f-4db8-9ef2-edbef1ddcac9 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:45.789516 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e60bc2-2a1f-4db8-9ef2-edbef1ddcac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:45.801355 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e60bc2-2a1f-4db8-9ef2-edbef1ddcac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:45.802763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7ad7b-ba89-4978-8c2e-1fbd27ae3130 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:45.802886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e60bc2-2a1f-4db8-9ef2-edbef1ddcac9 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-24e60bc2-2a1f-4db8-9ef2-edbef1ddcac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:45.817778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:45.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:45.820050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:45.820050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:45.820050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:45.820050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 528/1042] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:45.826241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e60bc2-2a1f-4db8-9ef2-edbef1ddcac9 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:45.841461 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ced32-381e-4587-a088-9ab4ec8bda82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:45.842072 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ced32-381e-4587-a088-9ab4ec8bda82 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:45.842355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ced32-381e-4587-a088-9ab4ec8bda82 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:45.842634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ced32-381e-4587-a088-9ab4ec8bda82 None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:45.843240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ced32-381e-4587-a088-9ab4ec8bda82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:45.844790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ced32-381e-4587-a088-9ab4ec8bda82 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:45.852347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 515/1043] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 24 07:33:45 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:33:45.939472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e60bc2-2a1f-4db8-9ef2-edbef1ddcac9 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:45.963008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ced32-381e-4587-a088-9ab4ec8bda82 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:45.978502 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32e13a-4201-4eb0-99df-d34afef1e7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:45.979221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32e13a-4201-4eb0-99df-d34afef1e7f5 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:45.979573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32e13a-4201-4eb0-99df-d34afef1e7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:45.979918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32e13a-4201-4eb0-99df-d34afef1e7f5 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:45.980572 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32e13a-4201-4eb0-99df-d34afef1e7f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:45.980921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32e13a-4201-4eb0-99df-d34afef1e7f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:45.982402 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32e13a-4201-4eb0-99df-d34afef1e7f5 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:45.996136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 529/1044] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:46.001937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32e13a-4201-4eb0-99df-d34afef1e7f5 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:46.005358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:46.076570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ced32-381e-4587-a088-9ab4ec8bda82 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:46.113162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32e13a-4201-4eb0-99df-d34afef1e7f5 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:46.116856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364feb8-f7f1-4520-a325-eb5ee104bbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:46.117495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364feb8-f7f1-4520-a325-eb5ee104bbd1 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:46.117778 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364feb8-f7f1-4520-a325-eb5ee104bbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:46.118052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364feb8-f7f1-4520-a325-eb5ee104bbd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:46.127259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:46.131475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364feb8-f7f1-4520-a325-eb5ee104bbd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-8364feb8-f7f1-4520-a325-eb5ee104bbd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:46.136237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:46.137360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:46.138293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:46.138293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:46.138293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:46.138293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:46.138293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:46.138293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:46.138293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:46.138293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:46.138293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:46.138293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:46.138293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:46.138293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:46.138293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 516/1045] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:33:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:46.143338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364feb8-f7f1-4520-a325-eb5ee104bbd1 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:46.152119 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaee85d-29e5-4f8f-ae5f-b5fde4f9b137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:46.152797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaee85d-29e5-4f8f-ae5f-b5fde4f9b137 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:46.153121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaee85d-29e5-4f8f-ae5f-b5fde4f9b137 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:46.153550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaee85d-29e5-4f8f-ae5f-b5fde4f9b137 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:46.161259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaee85d-29e5-4f8f-ae5f-b5fde4f9b137 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:46.162715 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaee85d-29e5-4f8f-ae5f-b5fde4f9b137 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-dfaee85d-29e5-4f8f-ae5f-b5fde4f9b137 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:46.174067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:46.175596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:46.176500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:46.176500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:46.176500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:46.176500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:46.176500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:46.176500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:46.176500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:46.176500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:46.176500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 530/1046] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:46.182533 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaee85d-29e5-4f8f-ae5f-b5fde4f9b137 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:46.252323 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:46.255493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364feb8-f7f1-4520-a325-eb5ee104bbd1 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:46.290784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724abcec-84f8-48f8-a2de-2e0c693dac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:46.291488 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724abcec-84f8-48f8-a2de-2e0c693dac35 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:46.291842 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724abcec-84f8-48f8-a2de-2e0c693dac35 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:46.292178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724abcec-84f8-48f8-a2de-2e0c693dac35 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:46.292943 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724abcec-84f8-48f8-a2de-2e0c693dac35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:46.293326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724abcec-84f8-48f8-a2de-2e0c693dac35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:46.294761 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724abcec-84f8-48f8-a2de-2e0c693dac35 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:46.296598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaee85d-29e5-4f8f-ae5f-b5fde4f9b137 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:46.304041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 517/1047] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:33:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:46.309773 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724abcec-84f8-48f8-a2de-2e0c693dac35 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:46.338225 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34372e-4146-415b-999f-5068dce04587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:46.338908 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34372e-4146-415b-999f-5068dce04587 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:46.339515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34372e-4146-415b-999f-5068dce04587 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:46.339793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34372e-4146-415b-999f-5068dce04587 None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:46.340927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34372e-4146-415b-999f-5068dce04587 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:46.341237 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34372e-4146-415b-999f-5068dce04587 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:46.343383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34372e-4146-415b-999f-5068dce04587 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:46.364721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 531/1048] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:46.366150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34372e-4146-415b-999f-5068dce04587 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:46.373333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:46.416201 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724abcec-84f8-48f8-a2de-2e0c693dac35 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:46.453034 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a85e1c-ae79-416d-8ba5-4af0832b9451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:46.454024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a85e1c-ae79-416d-8ba5-4af0832b9451 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:46.454367 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a85e1c-ae79-416d-8ba5-4af0832b9451 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:46.454862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a85e1c-ae79-416d-8ba5-4af0832b9451 None admin] PATH_INFO: `/v3/users/d78896376a7b4216ad5e2879ab46803e` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:46.470402 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a85e1c-ae79-416d-8ba5-4af0832b9451 None admin] RBAC: Authorizing `identity:get_user(user_id=d78896376a7b4216ad5e2879ab46803e)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:46.472944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a85e1c-ae79-416d-8ba5-4af0832b9451 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:46.479684 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 518/1049] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:33:46 2022] GET /identity/v3/users/d78896376a7b4216ad5e2879ab46803e => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:46.479828 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34372e-4146-415b-999f-5068dce04587 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:46.480690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a85e1c-ae79-416d-8ba5-4af0832b9451 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:46.501428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:46.515074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6814ca-7377-4cbd-b02a-88ccdae9e1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:46.515801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6814ca-7377-4cbd-b02a-88ccdae9e1da None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:46.516301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6814ca-7377-4cbd-b02a-88ccdae9e1da None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:46.516581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6814ca-7377-4cbd-b02a-88ccdae9e1da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:46.517359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6814ca-7377-4cbd-b02a-88ccdae9e1da None admin] RBAC: Adding query filter params (role.id=f126e5eef4964aa5af83192f79e8a1dd, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=bdf81f09006a42299c0807f77f16c748) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:46.517676 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6814ca-7377-4cbd-b02a-88ccdae9e1da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:46.519531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6814ca-7377-4cbd-b02a-88ccdae9e1da None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:46.525151 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 532/1050] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:46 2022] GET /identity/v3/role_assignments?user.id=bdf81f09006a42299c0807f77f16c748&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=f126e5eef4964aa5af83192f79e8a1dd => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:46.600202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a85e1c-ae79-416d-8ba5-4af0832b9451 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:46.623615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:46.639431 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84364acd-d99a-4f6b-9d81-0bb062fbfd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:46.640041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84364acd-d99a-4f6b-9d81-0bb062fbfd4c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:46.640318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84364acd-d99a-4f6b-9d81-0bb062fbfd4c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:46.640595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84364acd-d99a-4f6b-9d81-0bb062fbfd4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:46.714228 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84364acd-d99a-4f6b-9d81-0bb062fbfd4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:46.715758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84364acd-d99a-4f6b-9d81-0bb062fbfd4c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:46.820288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 519/1051] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:33:46.821935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6814ca-7377-4cbd-b02a-88ccdae9e1da None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:46.921765 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6814ca-7377-4cbd-b02a-88ccdae9e1da None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:46.958348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a533281-faa0-4bba-8d04-333860475b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:46.958983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a533281-faa0-4bba-8d04-333860475b24 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:46.959264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a533281-faa0-4bba-8d04-333860475b24 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:46.959537 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a533281-faa0-4bba-8d04-333860475b24 None admin] PATH_INFO: `/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:46.960671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a533281-faa0-4bba-8d04-333860475b24 None admin] RBAC: Authorizing `identity:get_project(project_id=8aba7cdcf0274d8b9b6530e4425ae914)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:46.962099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a533281-faa0-4bba-8d04-333860475b24 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:46.964634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 533/1052] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:33:46 2022] GET /identity/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:46.969759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84364acd-d99a-4f6b-9d81-0bb062fbfd4c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:46.991827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:47.074219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84364acd-d99a-4f6b-9d81-0bb062fbfd4c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:47.108064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64830612-2baa-42ab-8bcf-4c612629ae4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:47.108686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64830612-2baa-42ab-8bcf-4c612629ae4a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:47.108971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64830612-2baa-42ab-8bcf-4c612629ae4a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:47.109242 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64830612-2baa-42ab-8bcf-4c612629ae4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:47.109954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64830612-2baa-42ab-8bcf-4c612629ae4a None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=8aba7cdcf0274d8b9b6530e4425ae914, user.id=d78896376a7b4216ad5e2879ab46803e) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:47.110260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64830612-2baa-42ab-8bcf-4c612629ae4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:47.111666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64830612-2baa-42ab-8bcf-4c612629ae4a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:47.118148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 520/1053] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:46 2022] GET /identity/v3/role_assignments?user.id=d78896376a7b4216ad5e2879ab46803e&scope.project.id=8aba7cdcf0274d8b9b6530e4425ae914&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:47.776636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd04796-700d-43ea-99d4-6b411e25b3e2 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:47.777171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd04796-700d-43ea-99d4-6b411e25b3e2 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:47.777249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd04796-700d-43ea-99d4-6b411e25b3e2 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:47.778592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 534/1054] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:47.783696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045bcfe-585e-4ddb-a570-36d9df786a3b None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:47.783992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045bcfe-585e-4ddb-a570-36d9df786a3b None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:47.784264 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045bcfe-585e-4ddb-a570-36d9df786a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:47.802392 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-2045bcfe-585e-4ddb-a570-36d9df786a3b None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:47.803561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2045bcfe-585e-4ddb-a570-36d9df786a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:47.871342 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 521/1055] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:48.137242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd04796-700d-43ea-99d4-6b411e25b3e2 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:48.196617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:48.242928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd04796-700d-43ea-99d4-6b411e25b3e2 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:48.276695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113257ac-7a33-4c27-a85a-31b613721a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:48.277324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113257ac-7a33-4c27-a85a-31b613721a4b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:48.277605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113257ac-7a33-4c27-a85a-31b613721a4b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:48.277883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113257ac-7a33-4c27-a85a-31b613721a4b None admin] PATH_INFO: `/v3/services/image` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:48.278455 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113257ac-7a33-4c27-a85a-31b613721a4b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:48.279821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113257ac-7a33-4c27-a85a-31b613721a4b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-113257ac-7a33-4c27-a85a-31b613721a4b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:48.288551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:48.289549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:48.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:48.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:48.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:48.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:48.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:48.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:48.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:48.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:48.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:48.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:48.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:33:48.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:33:48.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:33:48.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:48.291896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 535/1056] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Aug 24 07:33:48 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:48.295509 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045bcfe-585e-4ddb-a570-36d9df786a3b None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:48.311152 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:48.395901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045bcfe-585e-4ddb-a570-36d9df786a3b None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:48.430634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8b8f9-6fdd-4dfe-b5a2-ce828b84f48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:48.431240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8b8f9-6fdd-4dfe-b5a2-ce828b84f48a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:48.431520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8b8f9-6fdd-4dfe-b5a2-ce828b84f48a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:48.431792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8b8f9-6fdd-4dfe-b5a2-ce828b84f48a None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:48.432416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8b8f9-6fdd-4dfe-b5a2-ce828b84f48a None admin] RBAC: Adding query filter params (name=image) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:48.432771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8b8f9-6fdd-4dfe-b5a2-ce828b84f48a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:48.434120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8b8f9-6fdd-4dfe-b5a2-ce828b84f48a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:48.434610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3f7bb-9659-4927-ae2f-e45fecf0fd6a None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:48.434902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3f7bb-9659-4927-ae2f-e45fecf0fd6a None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:48.435183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3f7bb-9659-4927-ae2f-e45fecf0fd6a None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:48.436523 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 536/1057] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:48.439466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 522/1058] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:48.442318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418e89a-d8b0-4271-a83c-72c3373c9b65 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:48.442603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418e89a-d8b0-4271-a83c-72c3373c9b65 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:48.442879 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418e89a-d8b0-4271-a83c-72c3373c9b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:48.445564 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8b8f9-6fdd-4dfe-b5a2-ce828b84f48a None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:48.466001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-3418e89a-d8b0-4271-a83c-72c3373c9b65 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:48.467182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3418e89a-d8b0-4271-a83c-72c3373c9b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:48.531511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:48.551851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 537/1059] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:48.555238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8b8f9-6fdd-4dfe-b5a2-ce828b84f48a None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:48.591358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30645f-6932-43a6-a83a-ff010a25439d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:48.591971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30645f-6932-43a6-a83a-ff010a25439d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:48.592253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30645f-6932-43a6-a83a-ff010a25439d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:48.595393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30645f-6932-43a6-a83a-ff010a25439d None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:48.595393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df30645f-6932-43a6-a83a-ff010a25439d None admin] RBAC: Adding query filter params (type=image) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:48.595393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df30645f-6932-43a6-a83a-ff010a25439d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:48.595393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df30645f-6932-43a6-a83a-ff010a25439d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:48.604968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 523/1060] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:48.606067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418e89a-d8b0-4271-a83c-72c3373c9b65 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:48.656444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:48.707780 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418e89a-d8b0-4271-a83c-72c3373c9b65 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:48.741884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b22d1-f939-447c-ad72-3ca9dfae3930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:48.742489 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b22d1-f939-447c-ad72-3ca9dfae3930 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:48.742766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b22d1-f939-447c-ad72-3ca9dfae3930 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:48.743133 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b22d1-f939-447c-ad72-3ca9dfae3930 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:48.743797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b22d1-f939-447c-ad72-3ca9dfae3930 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:48.745179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b22d1-f939-447c-ad72-3ca9dfae3930 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:48.826557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30645f-6932-43a6-a83a-ff010a25439d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:48.829661 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 538/1061] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:33:48 2022] POST /identity/v3/endpoints => generated 327 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:33:48.937761 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30645f-6932-43a6-a83a-ff010a25439d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:48.985046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a147a-c8fe-44f7-8460-913ed58b1117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:48.985674 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a147a-c8fe-44f7-8460-913ed58b1117 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:48.985950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a147a-c8fe-44f7-8460-913ed58b1117 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:48.986218 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a147a-c8fe-44f7-8460-913ed58b1117 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:48.993559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:48.997697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a147a-c8fe-44f7-8460-913ed58b1117 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-533a147a-c8fe-44f7-8460-913ed58b1117 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:49.003312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:49.004296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:49.007646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:49.007646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:49.007646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:49.007646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:49.007646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:49.007646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:49.007646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:49.007646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:49.007646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:49.007646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:49.007646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:33:49.007646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:49.007646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 524/1062] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Aug 24 07:33:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:49.010923 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b22d1-f939-447c-ad72-3ca9dfae3930 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:49.024043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf610ef-94b8-428a-bfde-54767e358c24 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:49.024326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf610ef-94b8-428a-bfde-54767e358c24 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:49.024725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf610ef-94b8-428a-bfde-54767e358c24 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:49.026714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 525/1063] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:49.031937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347b197-e6f7-488a-8884-54cb7f97a223 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:49.032441 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347b197-e6f7-488a-8884-54cb7f97a223 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:49.032780 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347b197-e6f7-488a-8884-54cb7f97a223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:49.051481 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-2347b197-e6f7-488a-8884-54cb7f97a223 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:49.052706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2347b197-e6f7-488a-8884-54cb7f97a223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:49.123820 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 526/1064] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:33:49.130281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b22d1-f939-447c-ad72-3ca9dfae3930 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:49.136672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:49.164444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0331464-6bd2-455d-aba1-a356fe18d024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:49.165189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0331464-6bd2-455d-aba1-a356fe18d024 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:49.165813 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0331464-6bd2-455d-aba1-a356fe18d024 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:49.166316 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0331464-6bd2-455d-aba1-a356fe18d024 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:49.167245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0331464-6bd2-455d-aba1-a356fe18d024 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:49.167733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0331464-6bd2-455d-aba1-a356fe18d024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:49.169613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0331464-6bd2-455d-aba1-a356fe18d024 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:49.180679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 539/1065] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Aug 24 07:33:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:49.184195 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347b197-e6f7-488a-8884-54cb7f97a223 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:49.287046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347b197-e6f7-488a-8884-54cb7f97a223 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:49.313602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:49.324387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cddf9-12aa-4ea2-a2b4-19e10dd60f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:49.325066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cddf9-12aa-4ea2-a2b4-19e10dd60f44 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:49.327887 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cddf9-12aa-4ea2-a2b4-19e10dd60f44 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:49.327887 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cddf9-12aa-4ea2-a2b4-19e10dd60f44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:49.335723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cddf9-12aa-4ea2-a2b4-19e10dd60f44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:49.337253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cddf9-12aa-4ea2-a2b4-19e10dd60f44 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-ee4cddf9-12aa-4ea2-a2b4-19e10dd60f44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:49.352373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:49.353548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:49.354869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:49.354869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:49.354869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:49.356990 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 527/1066] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:49.359227 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0331464-6bd2-455d-aba1-a356fe18d024 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:49.400862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cddf9-12aa-4ea2-a2b4-19e10dd60f44 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:49.466209 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0331464-6bd2-455d-aba1-a356fe18d024 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:49.501921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831a040-a6bc-4b87-8054-514ff535937d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:49.502724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831a040-a6bc-4b87-8054-514ff535937d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:49.502885 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831a040-a6bc-4b87-8054-514ff535937d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:49.503124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831a040-a6bc-4b87-8054-514ff535937d None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:49.503764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831a040-a6bc-4b87-8054-514ff535937d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:49.504063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831a040-a6bc-4b87-8054-514ff535937d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:49.505543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831a040-a6bc-4b87-8054-514ff535937d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:49.507240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cddf9-12aa-4ea2-a2b4-19e10dd60f44 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:49.522178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 540/1067] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:49.523269 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831a040-a6bc-4b87-8054-514ff535937d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:49.552514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:49.553940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cf092-4c38-4447-b2f0-a4432ac20b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:49.554885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cf092-4c38-4447-b2f0-a4432ac20b74 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:49.555176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cf092-4c38-4447-b2f0-a4432ac20b74 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:49.555698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cf092-4c38-4447-b2f0-a4432ac20b74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:49.566116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cf092-4c38-4447-b2f0-a4432ac20b74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-c57cf092-4c38-4447-b2f0-a4432ac20b74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:49.571459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:49.572618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:49.573592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:49.573592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:49.573592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:49.573592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:49.573592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:49.573592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:49.573592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:49.573592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:49.573592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:49.573592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:49.573592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:49.573592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:49.577872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 528/1068] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:33:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:49.578712 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cf092-4c38-4447-b2f0-a4432ac20b74 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:49.625193 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:49.635498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831a040-a6bc-4b87-8054-514ff535937d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:49.672981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccb947-46c1-46bb-b2e7-b56d3e9a59ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:49.673616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccb947-46c1-46bb-b2e7-b56d3e9a59ff None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:49.673902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccb947-46c1-46bb-b2e7-b56d3e9a59ff None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:49.674178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccb947-46c1-46bb-b2e7-b56d3e9a59ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:49.681381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccb947-46c1-46bb-b2e7-b56d3e9a59ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:49.682854 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccb947-46c1-46bb-b2e7-b56d3e9a59ff None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:49.684451 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cf092-4c38-4447-b2f0-a4432ac20b74 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-ebccb947-46c1-46bb-b2e7-b56d3e9a59ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:49.694651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:49.695610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:49.696582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:49.696582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:49.696582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:49.696582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:49.696582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:49.696582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:49.696582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:49.696582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:49.696582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:33:49.696582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:49.696582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 541/1069] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 24 07:33:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:49.701588 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccb947-46c1-46bb-b2e7-b56d3e9a59ff None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:49.723050 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf26652-2cce-43d7-a778-478de2867ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:49.724031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf26652-2cce-43d7-a778-478de2867ff3 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:49.724495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf26652-2cce-43d7-a778-478de2867ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:49.725017 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf26652-2cce-43d7-a778-478de2867ff3 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:49.725699 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf26652-2cce-43d7-a778-478de2867ff3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:49.726014 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf26652-2cce-43d7-a778-478de2867ff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:49.727388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf26652-2cce-43d7-a778-478de2867ff3 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:49.736213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 529/1070] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:33:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:49.742059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf26652-2cce-43d7-a778-478de2867ff3 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:49.794046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:49.810101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccb947-46c1-46bb-b2e7-b56d3e9a59ff None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:49.850049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb02f8-5934-4a09-bee8-234428801cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:49.850718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb02f8-5934-4a09-bee8-234428801cc1 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:49.850939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb02f8-5934-4a09-bee8-234428801cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:49.851215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb02f8-5934-4a09-bee8-234428801cc1 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:49.851948 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb02f8-5934-4a09-bee8-234428801cc1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:49.852258 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb02f8-5934-4a09-bee8-234428801cc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:49.853730 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb02f8-5934-4a09-bee8-234428801cc1 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:49.853954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf26652-2cce-43d7-a778-478de2867ff3 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:49.856056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:49.873134 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 542/1071] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 24 07:33:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:49.878983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb02f8-5934-4a09-bee8-234428801cc1 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:49.892690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e78493-e9de-4c32-851b-84023008b8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:49.894181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e78493-e9de-4c32-851b-84023008b8cf None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:49.894707 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e78493-e9de-4c32-851b-84023008b8cf None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:49.895164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e78493-e9de-4c32-851b-84023008b8cf None admin] PATH_INFO: `/v3/users/d78896376a7b4216ad5e2879ab46803e` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:49.896495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e78493-e9de-4c32-851b-84023008b8cf None admin] RBAC: Authorizing `identity:get_user(user_id=d78896376a7b4216ad5e2879ab46803e)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:49.898323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e78493-e9de-4c32-851b-84023008b8cf None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:49.900373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 530/1072] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:33:49 2022] GET /identity/v3/users/d78896376a7b4216ad5e2879ab46803e => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:49.905734 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e78493-e9de-4c32-851b-84023008b8cf None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:49.989845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb02f8-5934-4a09-bee8-234428801cc1 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:50.016956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e78493-e9de-4c32-851b-84023008b8cf None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:50.025216 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff64ff0-b88f-4390-9c9d-c654a7fc5c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:50.025851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff64ff0-b88f-4390-9c9d-c654a7fc5c9c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:50.026199 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff64ff0-b88f-4390-9c9d-c654a7fc5c9c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:50.026514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff64ff0-b88f-4390-9c9d-c654a7fc5c9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:50.040241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff64ff0-b88f-4390-9c9d-c654a7fc5c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:50.041759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff64ff0-b88f-4390-9c9d-c654a7fc5c9c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:50.045023 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:50.054031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f33d0-3f15-448a-b99a-b89bf3c448cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:50.054661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f33d0-3f15-448a-b99a-b89bf3c448cd None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:50.058757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f33d0-3f15-448a-b99a-b89bf3c448cd None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:50.058757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f33d0-3f15-448a-b99a-b89bf3c448cd None admin] PATH_INFO: `/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:50.058757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f33d0-3f15-448a-b99a-b89bf3c448cd None admin] RBAC: Authorizing `identity:get_project(project_id=8aba7cdcf0274d8b9b6530e4425ae914)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:50.058757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f33d0-3f15-448a-b99a-b89bf3c448cd None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-8ff64ff0-b88f-4390-9c9d-c654a7fc5c9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:50.059047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:50.065003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:50.066513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:50.066513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:50.066513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:50.066706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 531/1073] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:33:49 2022] GET /identity/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:50.066778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 543/1074] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:50.066846 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f33d0-3f15-448a-b99a-b89bf3c448cd None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:50.066915 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff64ff0-b88f-4390-9c9d-c654a7fc5c9c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:50.107729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:50.171421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f33d0-3f15-448a-b99a-b89bf3c448cd None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:50.172153 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff64ff0-b88f-4390-9c9d-c654a7fc5c9c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:50.206276 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7afd14-35a1-44bc-a1c5-48d3735c24ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:50.206750 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19446b5-4c12-4c79-ac36-70f974daab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:50.206900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7afd14-35a1-44bc-a1c5-48d3735c24ac None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:50.207148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7afd14-35a1-44bc-a1c5-48d3735c24ac None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:50.207408 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19446b5-4c12-4c79-ac36-70f974daab64 None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:50.207507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7afd14-35a1-44bc-a1c5-48d3735c24ac None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:50.207693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19446b5-4c12-4c79-ac36-70f974daab64 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:50.207969 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19446b5-4c12-4c79-ac36-70f974daab64 None admin] PATH_INFO: `/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914/users/d78896376a7b4216ad5e2879ab46803e/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:50.208066 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7afd14-35a1-44bc-a1c5-48d3735c24ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:50.208334 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7afd14-35a1-44bc-a1c5-48d3735c24ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:50.209721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7afd14-35a1-44bc-a1c5-48d3735c24ac None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:50.209916 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19446b5-4c12-4c79-ac36-70f974daab64 None admin] RBAC: Authorizing `identity:create_grant(project_id=8aba7cdcf0274d8b9b6530e4425ae914, user_id=d78896376a7b4216ad5e2879ab46803e, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:50.211335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19446b5-4c12-4c79-ac36-70f974daab64 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:50.222324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 544/1075] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:50.227822 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7afd14-35a1-44bc-a1c5-48d3735c24ac None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:50.275747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 532/1076] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:33:50 2022] PUT /identity/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914/users/d78896376a7b4216ad5e2879ab46803e/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:33:50.284710 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:50.335766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7afd14-35a1-44bc-a1c5-48d3735c24ac None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:50.373701 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344a457-dcbd-40d0-bf47-95fe6f2ebc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:50.374410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a457-dcbd-40d0-bf47-95fe6f2ebc0c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:50.374728 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a457-dcbd-40d0-bf47-95fe6f2ebc0c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:50.375008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a457-dcbd-40d0-bf47-95fe6f2ebc0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:50.384075 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344a457-dcbd-40d0-bf47-95fe6f2ebc0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:50.385671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344a457-dcbd-40d0-bf47-95fe6f2ebc0c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-6344a457-dcbd-40d0-bf47-95fe6f2ebc0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:50.398824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:50.399900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:50.400867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:50.400867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:50.400867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:50.400867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:50.400867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:50.400867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:50.400867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:50.400867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:50.400867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 545/1077] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:50.406088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19446b5-4c12-4c79-ac36-70f974daab64 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:50.510166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:50.514558 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19446b5-4c12-4c79-ac36-70f974daab64 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:50.555287 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c02b5-b7cf-46ef-9a28-527f28ec8e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:50.556260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c02b5-b7cf-46ef-9a28-527f28ec8e91 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:50.556796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c02b5-b7cf-46ef-9a28-527f28ec8e91 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:50.557299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c02b5-b7cf-46ef-9a28-527f28ec8e91 None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:50.558239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c02b5-b7cf-46ef-9a28-527f28ec8e91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:50.558795 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c02b5-b7cf-46ef-9a28-527f28ec8e91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:50.560442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c02b5-b7cf-46ef-9a28-527f28ec8e91 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:50.572881 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 533/1078] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:50.578774 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344a457-dcbd-40d0-bf47-95fe6f2ebc0c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:50.682579 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344a457-dcbd-40d0-bf47-95fe6f2ebc0c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:50.699038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:50.717508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aa6a3-12a3-4947-aa5b-38dd39521a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:50.718179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aa6a3-12a3-4947-aa5b-38dd39521a50 None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:50.718461 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aa6a3-12a3-4947-aa5b-38dd39521a50 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:50.718739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aa6a3-12a3-4947-aa5b-38dd39521a50 None admin] PATH_INFO: `/v3/projects/fc060764023e44a7ade80eeed98a235f/users/bdf81f09006a42299c0807f77f16c748/roles/f126e5eef4964aa5af83192f79e8a1dd` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:50.720896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26aa6a3-12a3-4947-aa5b-38dd39521a50 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc060764023e44a7ade80eeed98a235f, user_id=bdf81f09006a42299c0807f77f16c748, role_id=f126e5eef4964aa5af83192f79e8a1dd)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:50.723243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26aa6a3-12a3-4947-aa5b-38dd39521a50 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:50.794037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 546/1079] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:33:50 2022] PUT /identity/v3/projects/fc060764023e44a7ade80eeed98a235f/users/bdf81f09006a42299c0807f77f16c748/roles/f126e5eef4964aa5af83192f79e8a1dd => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:33:50.797967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66508067-b077-4d77-abf9-cb1fb25219b3 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:50.798260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66508067-b077-4d77-abf9-cb1fb25219b3 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:50.798535 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66508067-b077-4d77-abf9-cb1fb25219b3 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:50.800579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 534/1080] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:50.806266 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccb80d-4bad-4648-8377-0fd9e940ee80 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:50.806583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccb80d-4bad-4648-8377-0fd9e940ee80 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:50.806849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccb80d-4bad-4648-8377-0fd9e940ee80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:50.827488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-dfccb80d-4bad-4648-8377-0fd9e940ee80 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:50.828754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-dfccb80d-4bad-4648-8377-0fd9e940ee80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:50.904697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 547/1081] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:51.177559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66508067-b077-4d77-abf9-cb1fb25219b3 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:51.283720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66508067-b077-4d77-abf9-cb1fb25219b3 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:51.319504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd5014-48f6-425e-ae70-6aaf9d8b29da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:51.320105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd5014-48f6-425e-ae70-6aaf9d8b29da None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:51.320383 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd5014-48f6-425e-ae70-6aaf9d8b29da None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:51.320668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd5014-48f6-425e-ae70-6aaf9d8b29da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:51.347524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:51.389730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd5014-48f6-425e-ae70-6aaf9d8b29da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:51.391211 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd5014-48f6-425e-ae70-6aaf9d8b29da None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:51.490520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 535/1082] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:33:51.496081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccb80d-4bad-4648-8377-0fd9e940ee80 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:51.604768 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccb80d-4bad-4648-8377-0fd9e940ee80 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:51.612572 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:51.638826 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a780f-71d3-438e-8b76-fd6798402f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:51.639466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a780f-71d3-438e-8b76-fd6798402f58 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:51.640127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a780f-71d3-438e-8b76-fd6798402f58 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:51.640779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a780f-71d3-438e-8b76-fd6798402f58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:51.651181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a780f-71d3-438e-8b76-fd6798402f58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:51.652614 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a780f-71d3-438e-8b76-fd6798402f58 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-dc6a780f-71d3-438e-8b76-fd6798402f58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:51.668293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:51.669501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:51.670591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:51.670591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:51.670591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:51.672741 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 548/1083] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:51.679861 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd5014-48f6-425e-ae70-6aaf9d8b29da None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:51.738818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:51.782292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd5014-48f6-425e-ae70-6aaf9d8b29da None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:51.815954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef778e16-e305-4a66-819c-5daf1905048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:51.816765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef778e16-e305-4a66-819c-5daf1905048a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:51.817047 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef778e16-e305-4a66-819c-5daf1905048a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:51.817320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef778e16-e305-4a66-819c-5daf1905048a None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:51.817957 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef778e16-e305-4a66-819c-5daf1905048a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:51.818259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef778e16-e305-4a66-819c-5daf1905048a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:51.819621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef778e16-e305-4a66-819c-5daf1905048a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:51.840684 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 536/1084] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:52.227450 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d7261-3512-418b-bdaf-619665b5c9c7 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:52.227988 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d7261-3512-418b-bdaf-619665b5c9c7 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:52.228080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d7261-3512-418b-bdaf-619665b5c9c7 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:52.229400 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 549/1085] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:52.234801 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58e83e-bfc4-4154-b476-df96022c422c None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:52.235098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58e83e-bfc4-4154-b476-df96022c422c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:52.235379 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58e83e-bfc4-4154-b476-df96022c422c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:52.253101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-dc58e83e-bfc4-4154-b476-df96022c422c None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:52.254273 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-dc58e83e-bfc4-4154-b476-df96022c422c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:52.319907 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 537/1086] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:52.589448 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d7261-3512-418b-bdaf-619665b5c9c7 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:52.691171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d7261-3512-418b-bdaf-619665b5c9c7 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:52.725425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188b4f6-4c6f-430a-b922-3a9ffa1222d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:52.726055 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188b4f6-4c6f-430a-b922-3a9ffa1222d0 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:52.726338 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188b4f6-4c6f-430a-b922-3a9ffa1222d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:52.726613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188b4f6-4c6f-430a-b922-3a9ffa1222d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:52.736514 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81901c1-5d78-46ab-95ae-fe07aa638cc6 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:52.737589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188b4f6-4c6f-430a-b922-3a9ffa1222d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:52.737689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81901c1-5d78-46ab-95ae-fe07aa638cc6 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:52.737785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81901c1-5d78-46ab-95ae-fe07aa638cc6 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:52.739123 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 538/1087] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-2188b4f6-4c6f-430a-b922-3a9ffa1222d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:52.742048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:52.743000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:52.743840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:52.743840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:52.743840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:52.743840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:52.743840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:52.743840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:52.743840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:52.743840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:52.743840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:52.743840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:52.743840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:33:52.743840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:52.743840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 550/1088] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:33:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:52.744537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e9cdb-3ff5-4214-807c-3eb7a298db5a None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:52.744608 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e9cdb-3ff5-4214-807c-3eb7a298db5a None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:52.744932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e9cdb-3ff5-4214-807c-3eb7a298db5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:52.748950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188b4f6-4c6f-430a-b922-3a9ffa1222d0 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:52.763581 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-7b2e9cdb-3ff5-4214-807c-3eb7a298db5a None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:52.764800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-7b2e9cdb-3ff5-4214-807c-3eb7a298db5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:52.767355 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:52.808831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:52.831870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 539/1089] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:52.849396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188b4f6-4c6f-430a-b922-3a9ffa1222d0 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:52.881883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbe56a-d9f5-4187-82ea-ba70109d45f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:52.882475 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe56a-d9f5-4187-82ea-ba70109d45f0 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:52.882750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe56a-d9f5-4187-82ea-ba70109d45f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:52.883018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe56a-d9f5-4187-82ea-ba70109d45f0 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:52.883664 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbe56a-d9f5-4187-82ea-ba70109d45f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:52.883961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbe56a-d9f5-4187-82ea-ba70109d45f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:52.885332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbe56a-d9f5-4187-82ea-ba70109d45f0 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:52.893541 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 551/1090] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:33:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:52.899454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e9cdb-3ff5-4214-807c-3eb7a298db5a None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:52.995689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e9cdb-3ff5-4214-807c-3eb7a298db5a None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:53.028763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffef5b0-318f-4a15-b932-6ebdb6b531a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:53.029379 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef5b0-318f-4a15-b932-6ebdb6b531a4 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:53.029654 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef5b0-318f-4a15-b932-6ebdb6b531a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:53.029926 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef5b0-318f-4a15-b932-6ebdb6b531a4 None admin] PATH_INFO: `/v3/users/d78896376a7b4216ad5e2879ab46803e` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:53.031041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffef5b0-318f-4a15-b932-6ebdb6b531a4 None admin] RBAC: Authorizing `identity:get_user(user_id=d78896376a7b4216ad5e2879ab46803e)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:53.032454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffef5b0-318f-4a15-b932-6ebdb6b531a4 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:53.039044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 540/1091] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:33:52 2022] GET /identity/v3/users/d78896376a7b4216ad5e2879ab46803e => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:53.040206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbe56a-d9f5-4187-82ea-ba70109d45f0 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:53.100188 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffef5b0-318f-4a15-b932-6ebdb6b531a4 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:53.121553 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:53.141314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbe56a-d9f5-4187-82ea-ba70109d45f0 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:53.147228 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:53.176950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928aef2-fd4c-4dfb-82f5-e02d552f0390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:53.177556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928aef2-fd4c-4dfb-82f5-e02d552f0390 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:53.177836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928aef2-fd4c-4dfb-82f5-e02d552f0390 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:53.178106 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928aef2-fd4c-4dfb-82f5-e02d552f0390 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:53.198714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffef5b0-318f-4a15-b932-6ebdb6b531a4 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:53.233101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949984a-c57b-457e-bbb9-46cdac1014a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:53.233730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949984a-c57b-457e-bbb9-46cdac1014a9 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:53.234007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949984a-c57b-457e-bbb9-46cdac1014a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:53.234286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949984a-c57b-457e-bbb9-46cdac1014a9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:53.244369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949984a-c57b-457e-bbb9-46cdac1014a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:33:53.245497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928aef2-fd4c-4dfb-82f5-e02d552f0390 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:53.246952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928aef2-fd4c-4dfb-82f5-e02d552f0390 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-e949984a-c57b-457e-bbb9-46cdac1014a9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:33:53.249046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:53.250146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:53.251339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:53.251339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:53.251339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:53.251339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:53.251339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:33:53.251339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:33:53.251339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:33:53.251339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:33:53.251339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:33:53.251339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:33:53.251339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:33:53.251339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:53.251339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 541/1092] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Aug 24 07:33:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:53.256613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949984a-c57b-457e-bbb9-46cdac1014a9 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:53.331868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:53.351996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 552/1093] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:33:53.356715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949984a-c57b-457e-bbb9-46cdac1014a9 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:53.357823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928aef2-fd4c-4dfb-82f5-e02d552f0390 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:53.364892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:53.390656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57f79d-e46b-499b-a19c-bee858146318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:53.391264 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57f79d-e46b-499b-a19c-bee858146318 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:53.391544 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57f79d-e46b-499b-a19c-bee858146318 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:53.391818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57f79d-e46b-499b-a19c-bee858146318 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:53.392478 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57f79d-e46b-499b-a19c-bee858146318 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:53.392821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57f79d-e46b-499b-a19c-bee858146318 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:53.394547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57f79d-e46b-499b-a19c-bee858146318 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:53.407724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 542/1094] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Aug 24 07:33:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:53.409253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57f79d-e46b-499b-a19c-bee858146318 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:53.455057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928aef2-fd4c-4dfb-82f5-e02d552f0390 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:53.489549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae1ed9-943a-4df6-a740-be276c177a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:53.490173 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae1ed9-943a-4df6-a740-be276c177a46 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:53.490453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae1ed9-943a-4df6-a740-be276c177a46 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:53.490726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae1ed9-943a-4df6-a740-be276c177a46 None admin] PATH_INFO: `/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:53.491842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae1ed9-943a-4df6-a740-be276c177a46 None admin] RBAC: Authorizing `identity:get_project(project_id=8aba7cdcf0274d8b9b6530e4425ae914)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:53.493311 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae1ed9-943a-4df6-a740-be276c177a46 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:53.494950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 553/1095] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:33:53 2022] GET /identity/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:53.501094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae1ed9-943a-4df6-a740-be276c177a46 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:53.506951 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57f79d-e46b-499b-a19c-bee858146318 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:53.540792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0561ae-32cd-4b39-a036-a1c657c6bb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:53.541430 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0561ae-32cd-4b39-a036-a1c657c6bb86 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:53.541710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0561ae-32cd-4b39-a036-a1c657c6bb86 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:53.541985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0561ae-32cd-4b39-a036-a1c657c6bb86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:53.551110 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:53.553401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0561ae-32cd-4b39-a036-a1c657c6bb86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:53.554855 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0561ae-32cd-4b39-a036-a1c657c6bb86 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-8c0561ae-32cd-4b39-a036-a1c657c6bb86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:53.569110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:53.570090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:53.571064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:53.571064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:53.571064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:53.571064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 543/1096] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:33:53.575991 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0561ae-32cd-4b39-a036-a1c657c6bb86 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:53.600234 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:53.604539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae1ed9-943a-4df6-a740-be276c177a46 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:53.638151 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2274bf-fea7-424b-a839-a4a3623e7d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:53.638755 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2274bf-fea7-424b-a839-a4a3623e7d41 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:53.639039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2274bf-fea7-424b-a839-a4a3623e7d41 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:53.639313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2274bf-fea7-424b-a839-a4a3623e7d41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:53.640033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2274bf-fea7-424b-a839-a4a3623e7d41 None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=8aba7cdcf0274d8b9b6530e4425ae914, user.id=d78896376a7b4216ad5e2879ab46803e) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:53.640350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2274bf-fea7-424b-a839-a4a3623e7d41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:53.641803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2274bf-fea7-424b-a839-a4a3623e7d41 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:53.648025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 554/1097] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:53 2022] GET /identity/v3/role_assignments?user.id=d78896376a7b4216ad5e2879ab46803e&scope.project.id=8aba7cdcf0274d8b9b6530e4425ae914&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:53.678859 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0561ae-32cd-4b39-a036-a1c657c6bb86 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:53.715175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6cafe-e8ee-4c96-a798-94a1be5ea57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:53.715791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6cafe-e8ee-4c96-a798-94a1be5ea57d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:53.716071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6cafe-e8ee-4c96-a798-94a1be5ea57d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:53.716344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6cafe-e8ee-4c96-a798-94a1be5ea57d None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:53.717034 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6cafe-e8ee-4c96-a798-94a1be5ea57d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:53.717339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6cafe-e8ee-4c96-a798-94a1be5ea57d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:53.718719 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6cafe-e8ee-4c96-a798-94a1be5ea57d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:53.731898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 544/1098] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:53.738293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2274bf-fea7-424b-a839-a4a3623e7d41 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:53.847598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2274bf-fea7-424b-a839-a4a3623e7d41 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:53.860685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:53.881616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd74a4-9960-4a22-a138-0916d1f99d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:53.882238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd74a4-9960-4a22-a138-0916d1f99d2c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:53.882527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd74a4-9960-4a22-a138-0916d1f99d2c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:53.882867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd74a4-9960-4a22-a138-0916d1f99d2c None admin] PATH_INFO: `/v3/users/nova` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:53.890216 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd74a4-9960-4a22-a138-0916d1f99d2c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:53.891637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd74a4-9960-4a22-a138-0916d1f99d2c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-49fd74a4-9960-4a22-a138-0916d1f99d2c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:33:53.903612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:53.904825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:53.906313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:53.906313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:53.906313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:33:53.906313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:33:53.906313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:33:53.906313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:33:53.906313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:33:53.906313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:33:53.906313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:33:53.906313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:33:53.906313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 555/1099] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 24 07:33:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:53.911323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6cafe-e8ee-4c96-a798-94a1be5ea57d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:53.990196 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:54.009329 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6cafe-e8ee-4c96-a798-94a1be5ea57d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:54.043516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b617a98-f4de-4e17-89cf-8a3ab687c478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:54.044118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617a98-f4de-4e17-89cf-8a3ab687c478 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:54.044396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617a98-f4de-4e17-89cf-8a3ab687c478 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:54.044689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617a98-f4de-4e17-89cf-8a3ab687c478 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:54.045411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b617a98-f4de-4e17-89cf-8a3ab687c478 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:54.045717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b617a98-f4de-4e17-89cf-8a3ab687c478 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:54.047182 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b617a98-f4de-4e17-89cf-8a3ab687c478 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:54.063847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 545/1100] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 24 07:33:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:54.070113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd74a4-9960-4a22-a138-0916d1f99d2c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:54.168150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd74a4-9960-4a22-a138-0916d1f99d2c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:54.201010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6de81-cbb0-492a-a52b-e3f8abc56a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:54.201609 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6de81-cbb0-492a-a52b-e3f8abc56a4b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:54.201889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6de81-cbb0-492a-a52b-e3f8abc56a4b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:54.202161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6de81-cbb0-492a-a52b-e3f8abc56a4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:54.242020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:54.267016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a6de81-cbb0-492a-a52b-e3f8abc56a4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:54.269056 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a6de81-cbb0-492a-a52b-e3f8abc56a4b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:54.364359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 556/1101] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:33:54.370111 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b617a98-f4de-4e17-89cf-8a3ab687c478 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:54.465064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b617a98-f4de-4e17-89cf-8a3ab687c478 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:54.498035 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072a564-c649-4da3-af0f-ebb52d7768c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:54.498648 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072a564-c649-4da3-af0f-ebb52d7768c3 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:54.498927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072a564-c649-4da3-af0f-ebb52d7768c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:54.499226 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072a564-c649-4da3-af0f-ebb52d7768c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:54.506266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:54.509965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072a564-c649-4da3-af0f-ebb52d7768c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:54.511397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072a564-c649-4da3-af0f-ebb52d7768c3 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-f072a564-c649-4da3-af0f-ebb52d7768c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:33:54.525800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:54.526821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:33:54.527796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:33:54.527796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:33:54.527796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:54.528979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 546/1102] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:54.532693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6de81-cbb0-492a-a52b-e3f8abc56a4b None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:54.601698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:54.631043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6de81-cbb0-492a-a52b-e3f8abc56a4b None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:54.666467 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0859ef6-190b-4001-8c45-b78a1950bd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:54.667074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0859ef6-190b-4001-8c45-b78a1950bd80 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:54.667352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0859ef6-190b-4001-8c45-b78a1950bd80 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:54.667623 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0859ef6-190b-4001-8c45-b78a1950bd80 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:54.668261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0859ef6-190b-4001-8c45-b78a1950bd80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:54.668564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0859ef6-190b-4001-8c45-b78a1950bd80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:54.669941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0859ef6-190b-4001-8c45-b78a1950bd80 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:54.681460 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 557/1103] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:54.687164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072a564-c649-4da3-af0f-ebb52d7768c3 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:54.788046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072a564-c649-4da3-af0f-ebb52d7768c3 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:54.820472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cc051-7886-4bb1-8c0a-ff9cf9106e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:54.821122 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cc051-7886-4bb1-8c0a-ff9cf9106e97 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:54.821402 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cc051-7886-4bb1-8c0a-ff9cf9106e97 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:54.821675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cc051-7886-4bb1-8c0a-ff9cf9106e97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:54.828058 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cc051-7886-4bb1-8c0a-ff9cf9106e97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:54.829525 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cc051-7886-4bb1-8c0a-ff9cf9106e97 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-3e9cc051-7886-4bb1-8c0a-ff9cf9106e97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:33:54.841009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:33:54.841886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:33:54.842920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:33:54.842920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:33:54.842920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:33:54.842920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:33:54.842920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:33:54.842920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:33:54.842920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:33:54.842920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:33:54.842920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 547/1104] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:33:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:33:54.848473 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0859ef6-190b-4001-8c45-b78a1950bd80 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:54.945439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0859ef6-190b-4001-8c45-b78a1950bd80 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:54.973561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:54.978329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea686fd-259c-43bc-9aa1-140ae0833bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:54.978925 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea686fd-259c-43bc-9aa1-140ae0833bf0 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:54.979207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea686fd-259c-43bc-9aa1-140ae0833bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:54.979482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea686fd-259c-43bc-9aa1-140ae0833bf0 None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:54.980173 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea686fd-259c-43bc-9aa1-140ae0833bf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:54.980479 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea686fd-259c-43bc-9aa1-140ae0833bf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:54.981935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea686fd-259c-43bc-9aa1-140ae0833bf0 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:54.993754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 558/1105] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:33:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:54.999811 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cc051-7886-4bb1-8c0a-ff9cf9106e97 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:55.051638 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:55.100104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cc051-7886-4bb1-8c0a-ff9cf9106e97 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:55.132836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a43926-620f-4377-8c82-7bfb14ff350f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:55.133432 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a43926-620f-4377-8c82-7bfb14ff350f None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:55.133710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a43926-620f-4377-8c82-7bfb14ff350f None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:55.133981 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a43926-620f-4377-8c82-7bfb14ff350f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:55.134691 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a43926-620f-4377-8c82-7bfb14ff350f None admin] RBAC: Adding query filter params (role.id=f126e5eef4964aa5af83192f79e8a1dd, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=bdf81f09006a42299c0807f77f16c748) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:55.134996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a43926-620f-4377-8c82-7bfb14ff350f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:55.136407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a43926-620f-4377-8c82-7bfb14ff350f None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:55.142142 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 548/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:54 2022] GET /identity/v3/role_assignments?user.id=bdf81f09006a42299c0807f77f16c748&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=f126e5eef4964aa5af83192f79e8a1dd => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:33:55.604064 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508673f3-aa4c-4cf0-8c19-38bf27e5fffd None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:55.604599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508673f3-aa4c-4cf0-8c19-38bf27e5fffd None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:55.604599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508673f3-aa4c-4cf0-8c19-38bf27e5fffd None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:55.606054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 559/1107] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:55.611423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6990d26-e05e-42b4-9eb1-583eeab0106c None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:55.611720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6990d26-e05e-42b4-9eb1-583eeab0106c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:55.612575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6990d26-e05e-42b4-9eb1-583eeab0106c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:55.630753 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-c6990d26-e05e-42b4-9eb1-583eeab0106c None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:55.631969 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c6990d26-e05e-42b4-9eb1-583eeab0106c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:55.701277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 549/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:55.972331 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508673f3-aa4c-4cf0-8c19-38bf27e5fffd None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:56.068790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508673f3-aa4c-4cf0-8c19-38bf27e5fffd None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:56.101926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05320a-a26b-432e-ae41-aee27090ee48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:56.102531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05320a-a26b-432e-ae41-aee27090ee48 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:56.102805 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05320a-a26b-432e-ae41-aee27090ee48 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:56.103071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05320a-a26b-432e-ae41-aee27090ee48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:56.104154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05320a-a26b-432e-ae41-aee27090ee48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:56.105591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05320a-a26b-432e-ae41-aee27090ee48 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:56.107123 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 560/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:33:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:56.113380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6990d26-e05e-42b4-9eb1-583eeab0106c None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:56.156322 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:56.207580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6990d26-e05e-42b4-9eb1-583eeab0106c None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:56.239968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525bcf4-482e-4103-af22-636dca9c45e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:56.240493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525bcf4-482e-4103-af22-636dca9c45e4 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:56.240800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525bcf4-482e-4103-af22-636dca9c45e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:56.241075 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525bcf4-482e-4103-af22-636dca9c45e4 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:56.241711 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525bcf4-482e-4103-af22-636dca9c45e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:56.243130 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525bcf4-482e-4103-af22-636dca9c45e4 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:56.330895 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 550/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:33:56 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:33:58.250932 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e28d3-4f03-4efb-86a2-9685dc16f26d None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:58.251502 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e28d3-4f03-4efb-86a2-9685dc16f26d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:58.251502 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e28d3-4f03-4efb-86a2-9685dc16f26d None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:58.253012 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 561/1111] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:33:58.258122 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c3a73-bf8f-48e8-9857-699465b9e0d5 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:58.258521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c3a73-bf8f-48e8-9857-699465b9e0d5 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:58.258680 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c3a73-bf8f-48e8-9857-699465b9e0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:58.276922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-475c3a73-bf8f-48e8-9857-699465b9e0d5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:33:58.278104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-475c3a73-bf8f-48e8-9857-699465b9e0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:33:58.342694 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 551/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:33:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:33:58.610069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e28d3-4f03-4efb-86a2-9685dc16f26d None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:58.670257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:58.710739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e28d3-4f03-4efb-86a2-9685dc16f26d None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:58.745585 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e3b47-8f7d-45a5-85cd-0b3899bb0be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:58.746202 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e3b47-8f7d-45a5-85cd-0b3899bb0be7 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:58.746485 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e3b47-8f7d-45a5-85cd-0b3899bb0be7 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:58.746757 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e3b47-8f7d-45a5-85cd-0b3899bb0be7 None admin] PATH_INFO: `/v3/roles/8183d3986255476f959797314695f92a` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:58.748130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e3b47-8f7d-45a5-85cd-0b3899bb0be7 None admin] RBAC: Authorizing `identity:get_role(role_id=8183d3986255476f959797314695f92a)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:58.749529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e3b47-8f7d-45a5-85cd-0b3899bb0be7 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:58.750835 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 562/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:33:58 2022] GET /identity/v3/roles/8183d3986255476f959797314695f92a => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:58.756925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c3a73-bf8f-48e8-9857-699465b9e0d5 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:58.786630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:58.854166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c3a73-bf8f-48e8-9857-699465b9e0d5 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:58.886598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb25a1c-3b6a-4046-98c7-0363447258ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:58.887219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb25a1c-3b6a-4046-98c7-0363447258ac None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:58.887501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb25a1c-3b6a-4046-98c7-0363447258ac None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:58.887777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb25a1c-3b6a-4046-98c7-0363447258ac None admin] PATH_INFO: `/v3/users/1bf1f96a06204e8eb004d269cb4acb9c` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:58.901050 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb25a1c-3b6a-4046-98c7-0363447258ac None admin] RBAC: Authorizing `identity:get_user(user_id=1bf1f96a06204e8eb004d269cb4acb9c)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:58.902485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb25a1c-3b6a-4046-98c7-0363447258ac None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:58.904059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 552/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:33:58 2022] GET /identity/v3/users/1bf1f96a06204e8eb004d269cb4acb9c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:58.910644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e3b47-8f7d-45a5-85cd-0b3899bb0be7 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:59.006890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e3b47-8f7d-45a5-85cd-0b3899bb0be7 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:59.032633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:59.041147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b409717-2b25-43f2-9b08-bc3d69cac801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:59.041691 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b409717-2b25-43f2-9b08-bc3d69cac801 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:59.041974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b409717-2b25-43f2-9b08-bc3d69cac801 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:59.042287 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b409717-2b25-43f2-9b08-bc3d69cac801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:59.107549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b409717-2b25-43f2-9b08-bc3d69cac801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:59.109039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b409717-2b25-43f2-9b08-bc3d69cac801 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:59.268976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 563/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:33:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:33:59.274748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb25a1c-3b6a-4046-98c7-0363447258ac None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:59.367748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:59.379991 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb25a1c-3b6a-4046-98c7-0363447258ac None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:59.414123 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039d607-276c-416a-9903-faee4e8729f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:59.414757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039d607-276c-416a-9903-faee4e8729f4 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:59.415039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039d607-276c-416a-9903-faee4e8729f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:59.415315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039d607-276c-416a-9903-faee4e8729f4 None admin] PATH_INFO: `/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:59.416428 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039d607-276c-416a-9903-faee4e8729f4 None admin] RBAC: Authorizing `identity:get_project(project_id=8aba7cdcf0274d8b9b6530e4425ae914)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:59.417885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039d607-276c-416a-9903-faee4e8729f4 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:59.419490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 553/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:33:59 2022] GET /identity/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:33:59.425867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b409717-2b25-43f2-9b08-bc3d69cac801 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:33:59.453271 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:33:59.523493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b409717-2b25-43f2-9b08-bc3d69cac801 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:33:59.557435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029a5d7-d2d3-4859-870e-a4968f22c6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:33:59.558036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029a5d7-d2d3-4859-870e-a4968f22c6c6 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:33:59.558319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029a5d7-d2d3-4859-870e-a4968f22c6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:33:59.558595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029a5d7-d2d3-4859-870e-a4968f22c6c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:33:59.559312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029a5d7-d2d3-4859-870e-a4968f22c6c6 None admin] RBAC: Adding query filter params (role.id=8183d3986255476f959797314695f92a, scope.project.id=8aba7cdcf0274d8b9b6530e4425ae914, user.id=1bf1f96a06204e8eb004d269cb4acb9c) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:33:59.559664 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029a5d7-d2d3-4859-870e-a4968f22c6c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:33:59.561116 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029a5d7-d2d3-4859-870e-a4968f22c6c6 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:33:59.566965 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 564/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:33:59 2022] GET /identity/v3/role_assignments?user.id=1bf1f96a06204e8eb004d269cb4acb9c&scope.project.id=8aba7cdcf0274d8b9b6530e4425ae914&role.id=8183d3986255476f959797314695f92a => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:01.428059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97336ef1-69b4-44b9-ab96-95d0b1049949 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:01.428688 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97336ef1-69b4-44b9-ab96-95d0b1049949 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:01.428688 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97336ef1-69b4-44b9-ab96-95d0b1049949 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:01.429946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 554/1118] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:01.434937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2981d4-96f5-45d5-8979-53af3c079155 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:01.435399 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2981d4-96f5-45d5-8979-53af3c079155 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:01.435493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2981d4-96f5-45d5-8979-53af3c079155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:01.453375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-cf2981d4-96f5-45d5-8979-53af3c079155 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:01.454540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-cf2981d4-96f5-45d5-8979-53af3c079155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:01.519955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 565/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:34:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:34:01.784992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97336ef1-69b4-44b9-ab96-95d0b1049949 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:01.882486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97336ef1-69b4-44b9-ab96-95d0b1049949 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:01.915871 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47ca91-a784-4fa2-a7aa-6b35fca0c87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:01.916479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47ca91-a784-4fa2-a7aa-6b35fca0c87c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:01.916783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47ca91-a784-4fa2-a7aa-6b35fca0c87c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:01.917054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47ca91-a784-4fa2-a7aa-6b35fca0c87c None admin] PATH_INFO: `/v3/roles/8183d3986255476f959797314695f92a` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:01.918416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47ca91-a784-4fa2-a7aa-6b35fca0c87c None admin] RBAC: Authorizing `identity:get_role(role_id=8183d3986255476f959797314695f92a)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:01.919762 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47ca91-a784-4fa2-a7aa-6b35fca0c87c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:01.921078 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 555/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:34:01 2022] GET /identity/v3/roles/8183d3986255476f959797314695f92a => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:01.926845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2981d4-96f5-45d5-8979-53af3c079155 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:01.949340 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:02.021478 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2981d4-96f5-45d5-8979-53af3c079155 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:02.054041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc5ed7-7e89-417b-9b7a-8de284e2ca23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:02.054650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc5ed7-7e89-417b-9b7a-8de284e2ca23 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:02.054929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc5ed7-7e89-417b-9b7a-8de284e2ca23 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:02.055198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc5ed7-7e89-417b-9b7a-8de284e2ca23 None admin] PATH_INFO: `/v3/users/1bf1f96a06204e8eb004d269cb4acb9c` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:02.056292 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc5ed7-7e89-417b-9b7a-8de284e2ca23 None admin] RBAC: Authorizing `identity:get_user(user_id=1bf1f96a06204e8eb004d269cb4acb9c)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:02.057739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc5ed7-7e89-417b-9b7a-8de284e2ca23 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:02.059268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 566/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:34:01 2022] GET /identity/v3/users/1bf1f96a06204e8eb004d269cb4acb9c => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:02.065116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47ca91-a784-4fa2-a7aa-6b35fca0c87c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:02.090067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:02.162525 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47ca91-a784-4fa2-a7aa-6b35fca0c87c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:02.194919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2e03f-6c75-4f22-9966-d80544089773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:02.195592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2e03f-6c75-4f22-9966-d80544089773 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:02.195877 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2e03f-6c75-4f22-9966-d80544089773 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:02.196158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2e03f-6c75-4f22-9966-d80544089773 None admin] PATH_INFO: `/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:02.197253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2e03f-6c75-4f22-9966-d80544089773 None admin] RBAC: Authorizing `identity:get_project(project_id=8aba7cdcf0274d8b9b6530e4425ae914)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:02.198678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2e03f-6c75-4f22-9966-d80544089773 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:02.200268 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 556/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:34:02 2022] GET /identity/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:02.206682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc5ed7-7e89-417b-9b7a-8de284e2ca23 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:02.293663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:02.302579 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc5ed7-7e89-417b-9b7a-8de284e2ca23 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:02.335627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52c59f-0318-4dd6-9f81-fb6c5fdb6015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:02.336211 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52c59f-0318-4dd6-9f81-fb6c5fdb6015 None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:02.336497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52c59f-0318-4dd6-9f81-fb6c5fdb6015 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:02.336799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52c59f-0318-4dd6-9f81-fb6c5fdb6015 None admin] PATH_INFO: `/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914/users/1bf1f96a06204e8eb004d269cb4acb9c/roles/8183d3986255476f959797314695f92a` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:02.338904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52c59f-0318-4dd6-9f81-fb6c5fdb6015 None admin] RBAC: Authorizing `identity:create_grant(project_id=8aba7cdcf0274d8b9b6530e4425ae914, user_id=1bf1f96a06204e8eb004d269cb4acb9c, role_id=8183d3986255476f959797314695f92a)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:02.340322 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52c59f-0318-4dd6-9f81-fb6c5fdb6015 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:02.404901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 567/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:34:02 2022] PUT /identity/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914/users/1bf1f96a06204e8eb004d269cb4acb9c/roles/8183d3986255476f959797314695f92a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:34:04.297585 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d9911-d96b-4f83-abd1-e582db3b8e73 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:04.298177 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d9911-d96b-4f83-abd1-e582db3b8e73 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:04.298177 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d9911-d96b-4f83-abd1-e582db3b8e73 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:04.299614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 557/1124] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:04.304795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb9e32-a998-49bd-95ce-68c77664bcd0 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:04.305223 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb9e32-a998-49bd-95ce-68c77664bcd0 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:04.305351 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb9e32-a998-49bd-95ce-68c77664bcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:04.323052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a6cb9e32-a998-49bd-95ce-68c77664bcd0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:04.324227 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a6cb9e32-a998-49bd-95ce-68c77664bcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:04.388814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 568/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:34:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:34:04.656877 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d9911-d96b-4f83-abd1-e582db3b8e73 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:04.745828 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:04.754833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d9911-d96b-4f83-abd1-e582db3b8e73 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:04.786665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2dc5b-d1bc-458b-a409-434327fdb387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:04.787325 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2dc5b-d1bc-458b-a409-434327fdb387 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:04.787806 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2dc5b-d1bc-458b-a409-434327fdb387 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:04.788292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2dc5b-d1bc-458b-a409-434327fdb387 None admin] PATH_INFO: `/v3/roles/8183d3986255476f959797314695f92a` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:04.790114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2dc5b-d1bc-458b-a409-434327fdb387 None admin] RBAC: Authorizing `identity:get_role(role_id=8183d3986255476f959797314695f92a)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:04.791484 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2dc5b-d1bc-458b-a409-434327fdb387 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:04.792830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 558/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:34:04 2022] GET /identity/v3/roles/8183d3986255476f959797314695f92a => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:04.798584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb9e32-a998-49bd-95ce-68c77664bcd0 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:04.876271 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:04.894327 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb9e32-a998-49bd-95ce-68c77664bcd0 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:04.927707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636252e-d70b-4dd8-b08b-0b388d6ab6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:04.928258 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636252e-d70b-4dd8-b08b-0b388d6ab6ad None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:04.928540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636252e-d70b-4dd8-b08b-0b388d6ab6ad None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:04.928842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636252e-d70b-4dd8-b08b-0b388d6ab6ad None admin] PATH_INFO: `/v3/users/1bf1f96a06204e8eb004d269cb4acb9c` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:04.929943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636252e-d70b-4dd8-b08b-0b388d6ab6ad None admin] RBAC: Authorizing `identity:get_user(user_id=1bf1f96a06204e8eb004d269cb4acb9c)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:04.931362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636252e-d70b-4dd8-b08b-0b388d6ab6ad None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:04.932980 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 569/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:34:04 2022] GET /identity/v3/users/1bf1f96a06204e8eb004d269cb4acb9c => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:04.938696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2dc5b-d1bc-458b-a409-434327fdb387 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:05.035643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2dc5b-d1bc-458b-a409-434327fdb387 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:05.067925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d237b9-d7e1-4818-b20f-ea1dfb7992b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:05.068504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d237b9-d7e1-4818-b20f-ea1dfb7992b5 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:05.068860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d237b9-d7e1-4818-b20f-ea1dfb7992b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:05.069141 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d237b9-d7e1-4818-b20f-ea1dfb7992b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:05.095530 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:05.134481 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d237b9-d7e1-4818-b20f-ea1dfb7992b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:05.135923 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d237b9-d7e1-4818-b20f-ea1dfb7992b5 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:05.229516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 559/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:34:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:34:05.235423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636252e-d70b-4dd8-b08b-0b388d6ab6ad None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:05.331996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636252e-d70b-4dd8-b08b-0b388d6ab6ad None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:05.365561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc049c1-91a5-40dd-8194-c6e814d84d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:05.366176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc049c1-91a5-40dd-8194-c6e814d84d67 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:05.366458 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc049c1-91a5-40dd-8194-c6e814d84d67 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:05.366735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc049c1-91a5-40dd-8194-c6e814d84d67 None admin] PATH_INFO: `/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:05.367842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc049c1-91a5-40dd-8194-c6e814d84d67 None admin] RBAC: Authorizing `identity:get_project(project_id=8aba7cdcf0274d8b9b6530e4425ae914)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:05.369303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc049c1-91a5-40dd-8194-c6e814d84d67 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:05.370917 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 570/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:34:05 2022] GET /identity/v3/projects/8aba7cdcf0274d8b9b6530e4425ae914 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:05.376975 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d237b9-d7e1-4818-b20f-ea1dfb7992b5 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:05.434275 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:05.474042 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d237b9-d7e1-4818-b20f-ea1dfb7992b5 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:05.506486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbead0-7139-4327-ba4e-4ef65c709663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:05.507020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbead0-7139-4327-ba4e-4ef65c709663 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:05.507295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbead0-7139-4327-ba4e-4ef65c709663 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:05.507564 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbead0-7139-4327-ba4e-4ef65c709663 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:05.508274 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbead0-7139-4327-ba4e-4ef65c709663 None admin] RBAC: Adding query filter params (role.id=8183d3986255476f959797314695f92a, scope.project.id=8aba7cdcf0274d8b9b6530e4425ae914, user.id=1bf1f96a06204e8eb004d269cb4acb9c) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:34:05.508577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbead0-7139-4327-ba4e-4ef65c709663 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:05.510012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbead0-7139-4327-ba4e-4ef65c709663 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:05.515572 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 560/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:34:05 2022] GET /identity/v3/role_assignments?user.id=1bf1f96a06204e8eb004d269cb4acb9c&scope.project.id=8aba7cdcf0274d8b9b6530e4425ae914&role.id=8183d3986255476f959797314695f92a => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:07.412527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fe7b4-a62a-41c2-beed-a8b30eec0e64 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:07.413121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fe7b4-a62a-41c2-beed-a8b30eec0e64 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:07.413121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fe7b4-a62a-41c2-beed-a8b30eec0e64 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:07.414440 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 571/1131] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:07.419552 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77eeea-0b1e-41d8-8aca-5b2b5b323b4b None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:07.419932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77eeea-0b1e-41d8-8aca-5b2b5b323b4b None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:07.420107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77eeea-0b1e-41d8-8aca-5b2b5b323b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:07.438351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-bc77eeea-0b1e-41d8-8aca-5b2b5b323b4b None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:07.439519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-bc77eeea-0b1e-41d8-8aca-5b2b5b323b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:07.504943 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 561/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:34:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:34:07.769245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fe7b4-a62a-41c2-beed-a8b30eec0e64 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:07.787171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:07.866784 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fe7b4-a62a-41c2-beed-a8b30eec0e64 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:07.898690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadfad7-8542-4578-a412-6f3f8dd76f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:07.899296 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadfad7-8542-4578-a412-6f3f8dd76f47 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:07.899573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadfad7-8542-4578-a412-6f3f8dd76f47 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:07.899843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadfad7-8542-4578-a412-6f3f8dd76f47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:07.900954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadfad7-8542-4578-a412-6f3f8dd76f47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:07.902369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadfad7-8542-4578-a412-6f3f8dd76f47 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:07.903964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 572/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:34:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:34:07.910168 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77eeea-0b1e-41d8-8aca-5b2b5b323b4b None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:07.963341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:08.007925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77eeea-0b1e-41d8-8aca-5b2b5b323b4b None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:08.040336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bd0b8-2fbb-4b29-8160-ba91d8075efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:08.040899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bd0b8-2fbb-4b29-8160-ba91d8075efc None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:08.041181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bd0b8-2fbb-4b29-8160-ba91d8075efc None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:08.041453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bd0b8-2fbb-4b29-8160-ba91d8075efc None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:08.042097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bd0b8-2fbb-4b29-8160-ba91d8075efc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:08.043573 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bd0b8-2fbb-4b29-8160-ba91d8075efc None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:08.131867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 562/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 24 07:34:07 2022] POST /identity/v3/projects => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:34:10.049739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0c45e-e8e6-47e8-9c31-767f178e3fab None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:10.050310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0c45e-e8e6-47e8-9c31-767f178e3fab None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:10.050310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0c45e-e8e6-47e8-9c31-767f178e3fab None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:10.051666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 573/1135] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:10.056836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf7662-93e0-45c0-abc5-3f72f3818628 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:10.057234 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf7662-93e0-45c0-abc5-3f72f3818628 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:10.057385 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf7662-93e0-45c0-abc5-3f72f3818628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:10.075027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-efbf7662-93e0-45c0-abc5-3f72f3818628 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:10.076203 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-efbf7662-93e0-45c0-abc5-3f72f3818628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:10.140607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 563/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:34:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:34:10.406187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0c45e-e8e6-47e8-9c31-767f178e3fab None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:10.488911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:10.505123 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0c45e-e8e6-47e8-9c31-767f178e3fab None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:10.537984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad16d8-b403-4cdd-8d66-0acccd490294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:10.538614 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad16d8-b403-4cdd-8d66-0acccd490294 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:10.538896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad16d8-b403-4cdd-8d66-0acccd490294 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:10.539169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad16d8-b403-4cdd-8d66-0acccd490294 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:10.540267 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad16d8-b403-4cdd-8d66-0acccd490294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:10.541730 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad16d8-b403-4cdd-8d66-0acccd490294 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:10.543294 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 574/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:34:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:10.548992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf7662-93e0-45c0-abc5-3f72f3818628 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:10.582795 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:10.645733 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf7662-93e0-45c0-abc5-3f72f3818628 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:10.678591 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fe736-0ad4-4745-af69-2371bea2c5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:10.679119 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fe736-0ad4-4745-af69-2371bea2c5c2 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:10.679396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fe736-0ad4-4745-af69-2371bea2c5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:10.679668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fe736-0ad4-4745-af69-2371bea2c5c2 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:10.680297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fe736-0ad4-4745-af69-2371bea2c5c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:10.681748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fe736-0ad4-4745-af69-2371bea2c5c2 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:10.770421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 564/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:34:10 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:34:12.691996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334eba73-799f-4da3-917d-690bf33bcf18 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:12.692544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334eba73-799f-4da3-917d-690bf33bcf18 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:12.692544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334eba73-799f-4da3-917d-690bf33bcf18 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:12.693912 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 575/1139] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:12.699065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f79691-8426-45ec-b5bb-8ef89b5e1b30 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:12.699476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f79691-8426-45ec-b5bb-8ef89b5e1b30 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:12.699616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f79691-8426-45ec-b5bb-8ef89b5e1b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:12.717433 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-58f79691-8426-45ec-b5bb-8ef89b5e1b30 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:12.718600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-58f79691-8426-45ec-b5bb-8ef89b5e1b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:12.783648 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 565/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:34:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:34:13.052397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334eba73-799f-4da3-917d-690bf33bcf18 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:13.151644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334eba73-799f-4da3-917d-690bf33bcf18 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:13.184242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2eba44-3106-47dc-be32-5b71efc1d670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:13.184875 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eba44-3106-47dc-be32-5b71efc1d670 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:13.185151 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eba44-3106-47dc-be32-5b71efc1d670 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:13.185419 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eba44-3106-47dc-be32-5b71efc1d670 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:13.195667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2eba44-3106-47dc-be32-5b71efc1d670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-2d2eba44-3106-47dc-be32-5b71efc1d670 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:34:13.200229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:34:13.201432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:34:13.202542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:34:13.202542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:34:13.202542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:34:13.202542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:34:13.202542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:34:13.202542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:34:13.202542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:34:13.202542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:34:13.202542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:34:13.202542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:34:13.202542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:34:13.202542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:34:13.202542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 576/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:34:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:34:13.207557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f79691-8426-45ec-b5bb-8ef89b5e1b30 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:13.230467 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:13.304857 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f79691-8426-45ec-b5bb-8ef89b5e1b30 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:13.338013 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a6817-8072-4edd-b9aa-a362c6d1e346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:13.338599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a6817-8072-4edd-b9aa-a362c6d1e346 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:13.338875 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a6817-8072-4edd-b9aa-a362c6d1e346 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:13.339144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a6817-8072-4edd-b9aa-a362c6d1e346 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:13.339790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a6817-8072-4edd-b9aa-a362c6d1e346 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:34:13.340088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a6817-8072-4edd-b9aa-a362c6d1e346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:13.341472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a6817-8072-4edd-b9aa-a362c6d1e346 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:13.349865 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 566/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:34:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:13.355425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2eba44-3106-47dc-be32-5b71efc1d670 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:13.370723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:13.452034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2eba44-3106-47dc-be32-5b71efc1d670 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:13.484941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9961be9-7b8c-40f7-869c-486e57400b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:13.485551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9961be9-7b8c-40f7-869c-486e57400b36 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:13.485829 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9961be9-7b8c-40f7-869c-486e57400b36 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:13.486100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9961be9-7b8c-40f7-869c-486e57400b36 None admin] PATH_INFO: `/v3/users/2a3dabc4dc534fd08a6aa504ce671b71` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:13.499319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9961be9-7b8c-40f7-869c-486e57400b36 None admin] RBAC: Authorizing `identity:get_user(user_id=2a3dabc4dc534fd08a6aa504ce671b71)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:13.500770 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9961be9-7b8c-40f7-869c-486e57400b36 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:13.502477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 577/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:34:13 2022] GET /identity/v3/users/2a3dabc4dc534fd08a6aa504ce671b71 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:13.508388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a6817-8072-4edd-b9aa-a362c6d1e346 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:13.590503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:13.606493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a6817-8072-4edd-b9aa-a362c6d1e346 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:13.639441 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c32b29-4598-43df-9d7b-d3dbcef8fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:13.640036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c32b29-4598-43df-9d7b-d3dbcef8fcea None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:13.640320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c32b29-4598-43df-9d7b-d3dbcef8fcea None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:13.640598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c32b29-4598-43df-9d7b-d3dbcef8fcea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:13.705756 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c32b29-4598-43df-9d7b-d3dbcef8fcea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:13.707376 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c32b29-4598-43df-9d7b-d3dbcef8fcea None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:13.786300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:13.802302 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 567/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:34:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:34:13.808194 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9961be9-7b8c-40f7-869c-486e57400b36 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:13.874705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:13.903439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9961be9-7b8c-40f7-869c-486e57400b36 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:13.936071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fb294-5d86-4060-bfc6-43a58f2169d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:13.936706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fb294-5d86-4060-bfc6-43a58f2169d2 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:13.936985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fb294-5d86-4060-bfc6-43a58f2169d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:13.937252 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fb294-5d86-4060-bfc6-43a58f2169d2 None admin] PATH_INFO: `/v3/projects/d93bb1ea87ca4c98bf2dbdbbcc2643f2` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:13.938340 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fb294-5d86-4060-bfc6-43a58f2169d2 None admin] RBAC: Authorizing `identity:get_project(project_id=d93bb1ea87ca4c98bf2dbdbbcc2643f2)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:13.940332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fb294-5d86-4060-bfc6-43a58f2169d2 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:13.941958 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 578/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:34:13 2022] GET /identity/v3/projects/d93bb1ea87ca4c98bf2dbdbbcc2643f2 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:13.948016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c32b29-4598-43df-9d7b-d3dbcef8fcea None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:14.043030 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c32b29-4598-43df-9d7b-d3dbcef8fcea None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:14.076320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9116a3-9bc6-4b87-96b1-24f54e7b941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:14.076936 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9116a3-9bc6-4b87-96b1-24f54e7b941a None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:14.077214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9116a3-9bc6-4b87-96b1-24f54e7b941a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:14.077484 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9116a3-9bc6-4b87-96b1-24f54e7b941a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:14.078194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9116a3-9bc6-4b87-96b1-24f54e7b941a None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=d93bb1ea87ca4c98bf2dbdbbcc2643f2, user.id=2a3dabc4dc534fd08a6aa504ce671b71) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:34:14.078499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9116a3-9bc6-4b87-96b1-24f54e7b941a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:14.079905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9116a3-9bc6-4b87-96b1-24f54e7b941a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:14.085214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 568/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:34:13 2022] GET /identity/v3/role_assignments?user.id=2a3dabc4dc534fd08a6aa504ce671b71&scope.project.id=d93bb1ea87ca4c98bf2dbdbbcc2643f2&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:15.949311 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848285ea-3f59-47d6-93e0-b26d0fefac99 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:15.949851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848285ea-3f59-47d6-93e0-b26d0fefac99 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:15.949851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848285ea-3f59-47d6-93e0-b26d0fefac99 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:15.951187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 579/1147] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:15.956586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea95ca1-26ea-4138-ac65-dab48bff39cb None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:15.957044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea95ca1-26ea-4138-ac65-dab48bff39cb None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:15.957137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea95ca1-26ea-4138-ac65-dab48bff39cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:15.975344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-eea95ca1-26ea-4138-ac65-dab48bff39cb None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:15.976516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-eea95ca1-26ea-4138-ac65-dab48bff39cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:16.018560 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:16.042290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 569/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:34:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:34:16.311724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848285ea-3f59-47d6-93e0-b26d0fefac99 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:16.409707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848285ea-3f59-47d6-93e0-b26d0fefac99 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:16.442531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da81c1-1a5f-4847-a47c-c8ae557a2aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:16.443141 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da81c1-1a5f-4847-a47c-c8ae557a2aa8 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:16.443423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da81c1-1a5f-4847-a47c-c8ae557a2aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:16.443697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da81c1-1a5f-4847-a47c-c8ae557a2aa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:16.450184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:16.453686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da81c1-1a5f-4847-a47c-c8ae557a2aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-f6da81c1-1a5f-4847-a47c-c8ae557a2aa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:34:16.458393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:34:16.459631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:34:16.460766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:34:16.460766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:34:16.460766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:34:16.460766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:34:16.460766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:34:16.460766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:34:16.460766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:34:16.460766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:34:16.460766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:34:16.460766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:34:16.460766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:34:16.460766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:34:16.460766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 580/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:34:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:34:16.465434 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea95ca1-26ea-4138-ac65-dab48bff39cb None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:16.562446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea95ca1-26ea-4138-ac65-dab48bff39cb None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:16.595212 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0a290-78c2-40ff-9226-f74892089c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:16.596510 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0a290-78c2-40ff-9226-f74892089c20 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:16.597228 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0a290-78c2-40ff-9226-f74892089c20 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:16.597470 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0a290-78c2-40ff-9226-f74892089c20 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:16.598129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0a290-78c2-40ff-9226-f74892089c20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:34:16.598432 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0a290-78c2-40ff-9226-f74892089c20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:16.599789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0a290-78c2-40ff-9226-f74892089c20 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:16.607648 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 570/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:34:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:16.614011 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da81c1-1a5f-4847-a47c-c8ae557a2aa8 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:16.709998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da81c1-1a5f-4847-a47c-c8ae557a2aa8 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:16.744386 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8556999-0bf0-429f-98e9-f5416893c28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:16.745038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556999-0bf0-429f-98e9-f5416893c28c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:16.745321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556999-0bf0-429f-98e9-f5416893c28c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:16.745595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556999-0bf0-429f-98e9-f5416893c28c None admin] PATH_INFO: `/v3/users/2a3dabc4dc534fd08a6aa504ce671b71` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:16.746709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8556999-0bf0-429f-98e9-f5416893c28c None admin] RBAC: Authorizing `identity:get_user(user_id=2a3dabc4dc534fd08a6aa504ce671b71)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:16.748127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8556999-0bf0-429f-98e9-f5416893c28c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:16.749720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 581/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:34:16 2022] GET /identity/v3/users/2a3dabc4dc534fd08a6aa504ce671b71 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:16.755775 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0a290-78c2-40ff-9226-f74892089c20 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:16.798304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:16.856356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0a290-78c2-40ff-9226-f74892089c20 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:16.887776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395e1c7-7ab8-4ee0-b5cf-e051840e987b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:16.888392 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395e1c7-7ab8-4ee0-b5cf-e051840e987b None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:16.888694 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395e1c7-7ab8-4ee0-b5cf-e051840e987b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:16.888970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395e1c7-7ab8-4ee0-b5cf-e051840e987b None admin] PATH_INFO: `/v3/projects/d93bb1ea87ca4c98bf2dbdbbcc2643f2` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:16.890069 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395e1c7-7ab8-4ee0-b5cf-e051840e987b None admin] RBAC: Authorizing `identity:get_project(project_id=d93bb1ea87ca4c98bf2dbdbbcc2643f2)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:16.891487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395e1c7-7ab8-4ee0-b5cf-e051840e987b None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:16.893634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 571/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:34:16 2022] GET /identity/v3/projects/d93bb1ea87ca4c98bf2dbdbbcc2643f2 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:16.899602 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8556999-0bf0-429f-98e9-f5416893c28c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:16.963797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:16.997349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8556999-0bf0-429f-98e9-f5416893c28c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:17.029901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ae5b6-42de-4c76-9270-78c1ca3202a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:17.030544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ae5b6-42de-4c76-9270-78c1ca3202a7 None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:17.030827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ae5b6-42de-4c76-9270-78c1ca3202a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:17.031127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ae5b6-42de-4c76-9270-78c1ca3202a7 None admin] PATH_INFO: `/v3/projects/d93bb1ea87ca4c98bf2dbdbbcc2643f2/users/2a3dabc4dc534fd08a6aa504ce671b71/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:17.033126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ae5b6-42de-4c76-9270-78c1ca3202a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d93bb1ea87ca4c98bf2dbdbbcc2643f2, user_id=2a3dabc4dc534fd08a6aa504ce671b71, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:17.035038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ae5b6-42de-4c76-9270-78c1ca3202a7 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:17.108637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 582/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:34:16 2022] PUT /identity/v3/projects/d93bb1ea87ca4c98bf2dbdbbcc2643f2/users/2a3dabc4dc534fd08a6aa504ce671b71/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:34:18.974722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2cd5f-4dd3-48d9-b554-18ee52c458a1 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:18.975311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2cd5f-4dd3-48d9-b554-18ee52c458a1 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:18.975311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2cd5f-4dd3-48d9-b554-18ee52c458a1 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:18.976580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 572/1154] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:18.981747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c2767-91e3-48b8-96eb-ef2ebef94d6e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:18.982139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c2767-91e3-48b8-96eb-ef2ebef94d6e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:18.982304 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c2767-91e3-48b8-96eb-ef2ebef94d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:19.000311 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-ac2c2767-91e3-48b8-96eb-ef2ebef94d6e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:19.001521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ac2c2767-91e3-48b8-96eb-ef2ebef94d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:19.065391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 583/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:34:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:34:19.332941 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2cd5f-4dd3-48d9-b554-18ee52c458a1 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:19.430404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2cd5f-4dd3-48d9-b554-18ee52c458a1 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:19.439073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:19.464468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cd458-480e-4040-8350-0d1f63f42eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:19.465280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cd458-480e-4040-8350-0d1f63f42eab None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:19.465560 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cd458-480e-4040-8350-0d1f63f42eab None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:19.465832 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cd458-480e-4040-8350-0d1f63f42eab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:19.476178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cd458-480e-4040-8350-0d1f63f42eab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-ba9cd458-480e-4040-8350-0d1f63f42eab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:34:19.480827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:34:19.481921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:34:19.482949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:34:19.482949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:34:19.482949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:34:19.482949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:34:19.482949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:34:19.482949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:34:19.482949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:34:19.482949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:34:19.482949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:34:19.482949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:34:19.482949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:34:19.482949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:34:19.482949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 573/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:34:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:34:19.487860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c2767-91e3-48b8-96eb-ef2ebef94d6e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:19.541883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:19.583802 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c2767-91e3-48b8-96eb-ef2ebef94d6e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:19.617270 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93a8b6-03b9-4815-b8d1-b8d8fa3ffabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:19.618352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93a8b6-03b9-4815-b8d1-b8d8fa3ffabc None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:19.618633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93a8b6-03b9-4815-b8d1-b8d8fa3ffabc None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:19.618900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93a8b6-03b9-4815-b8d1-b8d8fa3ffabc None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:19.619550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93a8b6-03b9-4815-b8d1-b8d8fa3ffabc None admin] RBAC: Adding query filter params (name=admin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:34:19.619849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93a8b6-03b9-4815-b8d1-b8d8fa3ffabc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:19.621289 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93a8b6-03b9-4815-b8d1-b8d8fa3ffabc None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:19.629597 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 584/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:34:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:19.635357 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cd458-480e-4040-8350-0d1f63f42eab None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:19.733084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cd458-480e-4040-8350-0d1f63f42eab None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:19.765662 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaaffcd-5d56-48bf-a9a0-aa651c199b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:19.766266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaffcd-5d56-48bf-a9a0-aa651c199b1c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:19.766548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaffcd-5d56-48bf-a9a0-aa651c199b1c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:19.766821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaffcd-5d56-48bf-a9a0-aa651c199b1c None admin] PATH_INFO: `/v3/users/2a3dabc4dc534fd08a6aa504ce671b71` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:19.768426 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaffcd-5d56-48bf-a9a0-aa651c199b1c None admin] RBAC: Authorizing `identity:get_user(user_id=2a3dabc4dc534fd08a6aa504ce671b71)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:19.769877 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaffcd-5d56-48bf-a9a0-aa651c199b1c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:19.771413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 574/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:34:19 2022] GET /identity/v3/users/2a3dabc4dc534fd08a6aa504ce671b71 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:19.777244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93a8b6-03b9-4815-b8d1-b8d8fa3ffabc None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:19.871974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93a8b6-03b9-4815-b8d1-b8d8fa3ffabc None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:19.888484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:19.904467 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded780e4-a2bf-4b57-b589-d2940a333e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:19.905087 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded780e4-a2bf-4b57-b589-d2940a333e53 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:19.905368 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded780e4-a2bf-4b57-b589-d2940a333e53 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:19.905641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded780e4-a2bf-4b57-b589-d2940a333e53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:19.969672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded780e4-a2bf-4b57-b589-d2940a333e53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:19.971193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded780e4-a2bf-4b57-b589-d2940a333e53 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:20.065180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 585/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:34:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:34:20.072087 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaaffcd-5d56-48bf-a9a0-aa651c199b1c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:20.106816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:20.169493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaaffcd-5d56-48bf-a9a0-aa651c199b1c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:20.201914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4564d5f-1c9d-4d38-9428-c5dcfe0c45b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:20.202530 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4564d5f-1c9d-4d38-9428-c5dcfe0c45b8 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:20.202812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4564d5f-1c9d-4d38-9428-c5dcfe0c45b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:20.203092 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4564d5f-1c9d-4d38-9428-c5dcfe0c45b8 None admin] PATH_INFO: `/v3/projects/d93bb1ea87ca4c98bf2dbdbbcc2643f2` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:20.204185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4564d5f-1c9d-4d38-9428-c5dcfe0c45b8 None admin] RBAC: Authorizing `identity:get_project(project_id=d93bb1ea87ca4c98bf2dbdbbcc2643f2)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:20.206403 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4564d5f-1c9d-4d38-9428-c5dcfe0c45b8 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:20.208003 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 575/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:34:20 2022] GET /identity/v3/projects/d93bb1ea87ca4c98bf2dbdbbcc2643f2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:20.214036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded780e4-a2bf-4b57-b589-d2940a333e53 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:20.231963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:20.310122 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded780e4-a2bf-4b57-b589-d2940a333e53 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:20.343183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e474c-c58a-410c-b665-7796595c62f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:20.343765 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e474c-c58a-410c-b665-7796595c62f6 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:20.344040 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e474c-c58a-410c-b665-7796595c62f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:20.344309 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e474c-c58a-410c-b665-7796595c62f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:20.345059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e474c-c58a-410c-b665-7796595c62f6 None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=d93bb1ea87ca4c98bf2dbdbbcc2643f2, user.id=2a3dabc4dc534fd08a6aa504ce671b71) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:34:20.345367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e474c-c58a-410c-b665-7796595c62f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:20.347233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e474c-c58a-410c-b665-7796595c62f6 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:20.352634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 586/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:34:20 2022] GET /identity/v3/role_assignments?user.id=2a3dabc4dc534fd08a6aa504ce671b71&scope.project.id=d93bb1ea87ca4c98bf2dbdbbcc2643f2&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:22.253147 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d21a7c-5f37-4de0-948a-b14a0c070a5a None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:22.253689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d21a7c-5f37-4de0-948a-b14a0c070a5a None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:22.253689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d21a7c-5f37-4de0-948a-b14a0c070a5a None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:22.255024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 576/1162] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:22.260206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1b6de-3e4a-4b79-8a69-f1339a1be302 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:22.260611 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1b6de-3e4a-4b79-8a69-f1339a1be302 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:22.260819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1b6de-3e4a-4b79-8a69-f1339a1be302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:22.279186 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-cbe1b6de-3e4a-4b79-8a69-f1339a1be302 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:22.280369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-cbe1b6de-3e4a-4b79-8a69-f1339a1be302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:22.344792 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 587/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:34:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:34:22.614978 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d21a7c-5f37-4de0-948a-b14a0c070a5a None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:22.713420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d21a7c-5f37-4de0-948a-b14a0c070a5a None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:22.715641 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:22.747688 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798291e-e6f1-48e8-b074-8e081f2e06ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:22.748285 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798291e-e6f1-48e8-b074-8e081f2e06ac None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:22.748565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798291e-e6f1-48e8-b074-8e081f2e06ac None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:22.748868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798291e-e6f1-48e8-b074-8e081f2e06ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:22.813130 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798291e-e6f1-48e8-b074-8e081f2e06ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:22.814723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798291e-e6f1-48e8-b074-8e081f2e06ac None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:22.909262 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 577/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:34:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:34:22.915168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1b6de-3e4a-4b79-8a69-f1339a1be302 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:22.928780 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:23.010627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1b6de-3e4a-4b79-8a69-f1339a1be302 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:23.044027 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae2e2d-a036-464c-996e-c102f7faef9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:23.044639 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae2e2d-a036-464c-996e-c102f7faef9b None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:23.044944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae2e2d-a036-464c-996e-c102f7faef9b None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:23.045215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae2e2d-a036-464c-996e-c102f7faef9b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:23.051922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae2e2d-a036-464c-996e-c102f7faef9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:23.053382 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae2e2d-a036-464c-996e-c102f7faef9b None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-a0ae2e2d-a036-464c-996e-c102f7faef9b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:34:23.064161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:34:23.065297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:34:23.066649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 588/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 24 07:34:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:34:23.071521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798291e-e6f1-48e8-b074-8e081f2e06ac None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:23.074679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:23.168056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798291e-e6f1-48e8-b074-8e081f2e06ac None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:23.202032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb3e86-f4c5-4c90-922f-f9b62b4f4504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:23.202630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb3e86-f4c5-4c90-922f-f9b62b4f4504 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:23.202910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb3e86-f4c5-4c90-922f-f9b62b4f4504 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:23.203184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb3e86-f4c5-4c90-922f-f9b62b4f4504 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:23.203815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb3e86-f4c5-4c90-922f-f9b62b4f4504 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:34:23.204120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb3e86-f4c5-4c90-922f-f9b62b4f4504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:23.205751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb3e86-f4c5-4c90-922f-f9b62b4f4504 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:23.212602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 578/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Aug 24 07:34:23 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:25.081109 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3695a7-0a63-4665-93ab-e13e76ba3924 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:25.081681 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3695a7-0a63-4665-93ab-e13e76ba3924 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:25.081681 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3695a7-0a63-4665-93ab-e13e76ba3924 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:25.082985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 589/1167] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:25.088122 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e328fc-dfb4-4876-8c3d-985b5e0951af None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:25.088496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e328fc-dfb4-4876-8c3d-985b5e0951af None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:25.088709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e328fc-dfb4-4876-8c3d-985b5e0951af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:25.106635 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-b3e328fc-dfb4-4876-8c3d-985b5e0951af None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:25.107807 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b3e328fc-dfb4-4876-8c3d-985b5e0951af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:25.167607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:25.172868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 579/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:34:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:34:25.438308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3695a7-0a63-4665-93ab-e13e76ba3924 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:25.507857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:25.536960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3695a7-0a63-4665-93ab-e13e76ba3924 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:25.570862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471aca1-931b-4b3b-9774-e429d42525f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:25.571461 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471aca1-931b-4b3b-9774-e429d42525f0 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:25.571744 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471aca1-931b-4b3b-9774-e429d42525f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:25.572016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471aca1-931b-4b3b-9774-e429d42525f0 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:25.572796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471aca1-931b-4b3b-9774-e429d42525f0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:25.574155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471aca1-931b-4b3b-9774-e429d42525f0 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:25.669400 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 590/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Wed Aug 24 07:34:25 2022] POST /identity/v3/domains => generated 267 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:34:27.572875 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abfdf2-90cf-41ed-816a-59727483ae31 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:27.573347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abfdf2-90cf-41ed-816a-59727483ae31 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:27.573438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abfdf2-90cf-41ed-816a-59727483ae31 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:27.574747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 580/1170] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:27.580495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f898f-f8df-4660-adfd-be9a8099c73c None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:27.580992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f898f-f8df-4660-adfd-be9a8099c73c None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:27.581273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f898f-f8df-4660-adfd-be9a8099c73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:27.599710 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-8c9f898f-f8df-4660-adfd-be9a8099c73c None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:27.600908 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8c9f898f-f8df-4660-adfd-be9a8099c73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:27.665012 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 591/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:34:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:34:27.930354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abfdf2-90cf-41ed-816a-59727483ae31 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:28.029361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abfdf2-90cf-41ed-816a-59727483ae31 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:28.062275 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf738d-cfc8-497b-9ccb-fe465d213099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:28.063062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf738d-cfc8-497b-9ccb-fe465d213099 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:28.063369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf738d-cfc8-497b-9ccb-fe465d213099 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:28.063641 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf738d-cfc8-497b-9ccb-fe465d213099 None admin] PATH_INFO: `/v3/domains/f6fdd1294d4d4da5816ecaa90c2a1f4d` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:28.074498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf738d-cfc8-497b-9ccb-fe465d213099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f6fdd1294d4d4da5816ecaa90c2a1f4d)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:28.075936 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf738d-cfc8-497b-9ccb-fe465d213099 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:28.077519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 581/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Wed Aug 24 07:34:27 2022] GET /identity/v3/domains/f6fdd1294d4d4da5816ecaa90c2a1f4d => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:28.083621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f898f-f8df-4660-adfd-be9a8099c73c None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:28.143404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:28.180158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f898f-f8df-4660-adfd-be9a8099c73c None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:28.213500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f76ab3-5b72-46d3-9f96-3d53ec23ae7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:28.214043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f76ab3-5b72-46d3-9f96-3d53ec23ae7c None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:28.214330 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f76ab3-5b72-46d3-9f96-3d53ec23ae7c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:28.214630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f76ab3-5b72-46d3-9f96-3d53ec23ae7c None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:28.215260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f76ab3-5b72-46d3-9f96-3d53ec23ae7c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:28.216713 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f76ab3-5b72-46d3-9f96-3d53ec23ae7c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:28.303570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 592/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 24 07:34:28 2022] POST /identity/v3/projects => generated 361 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:34:30.229270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c7aad-5199-4d16-88eb-5e78de23afbe None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:30.229828 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c7aad-5199-4d16-88eb-5e78de23afbe None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:30.229828 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c7aad-5199-4d16-88eb-5e78de23afbe None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:30.231158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 582/1174] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:30.236523 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db029ed0-11b0-4e0a-9ff4-c4872fa639f9 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:30.237057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db029ed0-11b0-4e0a-9ff4-c4872fa639f9 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:30.237832 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db029ed0-11b0-4e0a-9ff4-c4872fa639f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:30.255469 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-db029ed0-11b0-4e0a-9ff4-c4872fa639f9 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:30.256777 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-db029ed0-11b0-4e0a-9ff4-c4872fa639f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:30.321389 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 593/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:34:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:34:30.609288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c7aad-5199-4d16-88eb-5e78de23afbe None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:30.667308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:30.710077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c7aad-5199-4d16-88eb-5e78de23afbe None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:30.742829 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea3987-c026-495f-a515-a713c925fc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:30.743447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea3987-c026-495f-a515-a713c925fc63 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:30.743727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea3987-c026-495f-a515-a713c925fc63 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:30.744003 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea3987-c026-495f-a515-a713c925fc63 None admin] PATH_INFO: `/v3/domains/f6fdd1294d4d4da5816ecaa90c2a1f4d` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:30.745128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea3987-c026-495f-a515-a713c925fc63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f6fdd1294d4d4da5816ecaa90c2a1f4d)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:30.746560 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea3987-c026-495f-a515-a713c925fc63 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:30.748105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 583/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Wed Aug 24 07:34:30 2022] GET /identity/v3/domains/f6fdd1294d4d4da5816ecaa90c2a1f4d => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:30.754498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db029ed0-11b0-4e0a-9ff4-c4872fa639f9 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:30.798071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:30.850391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db029ed0-11b0-4e0a-9ff4-c4872fa639f9 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:30.883814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7da3d-5aa3-44de-b9a3-e3c76efce9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:30.884409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7da3d-5aa3-44de-b9a3-e3c76efce9c2 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:30.884718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7da3d-5aa3-44de-b9a3-e3c76efce9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:30.884997 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7da3d-5aa3-44de-b9a3-e3c76efce9c2 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:30.885638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7da3d-5aa3-44de-b9a3-e3c76efce9c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:30.887059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7da3d-5aa3-44de-b9a3-e3c76efce9c2 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:30.981603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 594/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:34:30 2022] POST /identity/v3/users => generated 311 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:34:32.907495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10041d42-1db5-4bb8-963d-24e39f8f0207 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:32.908064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10041d42-1db5-4bb8-963d-24e39f8f0207 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:32.908064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10041d42-1db5-4bb8-963d-24e39f8f0207 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:32.910393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 584/1178] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:34:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:32.915556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a9e49-6ce7-4905-bb0b-da997b73b821 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:32.915991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a9e49-6ce7-4905-bb0b-da997b73b821 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:32.916118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a9e49-6ce7-4905-bb0b-da997b73b821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:32.934640 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-d77a9e49-6ce7-4905-bb0b-da997b73b821 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:32.935827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d77a9e49-6ce7-4905-bb0b-da997b73b821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:33.000798 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 595/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:34:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:34:33.267417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10041d42-1db5-4bb8-963d-24e39f8f0207 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:33.368109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10041d42-1db5-4bb8-963d-24e39f8f0207 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:33.392983 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:33.401686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428e791-2fdf-48d9-b37a-08824406b45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:33.403614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428e791-2fdf-48d9-b37a-08824406b45d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:33.403614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428e791-2fdf-48d9-b37a-08824406b45d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:33.403614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428e791-2fdf-48d9-b37a-08824406b45d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:33.413240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428e791-2fdf-48d9-b37a-08824406b45d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-a428e791-2fdf-48d9-b37a-08824406b45d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:34:33.417868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:34:33.419112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:34:33.420216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:34:33.420216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:34:33.420216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:34:33.420216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:34:33.420216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:34:33.420216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:34:33.420216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:34:33.420216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:34:33.420216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:34:33.420216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:34:33.420216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:34:33.420216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:34:33.420216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 585/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:34:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:34:33.425168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a9e49-6ce7-4905-bb0b-da997b73b821 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:33.453544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:33.521193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a9e49-6ce7-4905-bb0b-da997b73b821 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:33.554657 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63868cf1-cfd5-43bc-a7ee-4c4f2365c1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:33.555256 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63868cf1-cfd5-43bc-a7ee-4c4f2365c1a2 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:33.555535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63868cf1-cfd5-43bc-a7ee-4c4f2365c1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:33.555809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63868cf1-cfd5-43bc-a7ee-4c4f2365c1a2 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:33.556463 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63868cf1-cfd5-43bc-a7ee-4c4f2365c1a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:34:33.556795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63868cf1-cfd5-43bc-a7ee-4c4f2365c1a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:33.558149 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63868cf1-cfd5-43bc-a7ee-4c4f2365c1a2 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:33.566236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 596/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:34:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:33.572135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428e791-2fdf-48d9-b37a-08824406b45d None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:33.670923 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428e791-2fdf-48d9-b37a-08824406b45d None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:33.703722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec36baf-f5f5-4859-820d-68a161b3b89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:33.704340 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec36baf-f5f5-4859-820d-68a161b3b89f None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:33.704624 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec36baf-f5f5-4859-820d-68a161b3b89f None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:33.704929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec36baf-f5f5-4859-820d-68a161b3b89f None admin] PATH_INFO: `/v3/users/37efde09728b4813b1a4a2650e1939c2` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:33.718174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec36baf-f5f5-4859-820d-68a161b3b89f None admin] RBAC: Authorizing `identity:get_user(user_id=37efde09728b4813b1a4a2650e1939c2)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:33.719624 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec36baf-f5f5-4859-820d-68a161b3b89f None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:33.721251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 586/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:34:33 2022] GET /identity/v3/users/37efde09728b4813b1a4a2650e1939c2 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:33.727612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63868cf1-cfd5-43bc-a7ee-4c4f2365c1a2 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:33.820269 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:33.828541 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63868cf1-cfd5-43bc-a7ee-4c4f2365c1a2 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:33.863145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d70c7-8c04-4b14-8402-8e47c6195711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:33.863739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d70c7-8c04-4b14-8402-8e47c6195711 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:33.864022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d70c7-8c04-4b14-8402-8e47c6195711 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:33.864292 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d70c7-8c04-4b14-8402-8e47c6195711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:33.928415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d70c7-8c04-4b14-8402-8e47c6195711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:33.929891 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d70c7-8c04-4b14-8402-8e47c6195711 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:34.015095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:34.024349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 597/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:34:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:34:34.030338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec36baf-f5f5-4859-820d-68a161b3b89f None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:34.075042 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:34.129110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec36baf-f5f5-4859-820d-68a161b3b89f None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:34.163323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a76e3-5156-48cb-9156-44af3ed6d950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:34.163939 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a76e3-5156-48cb-9156-44af3ed6d950 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:34.164222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a76e3-5156-48cb-9156-44af3ed6d950 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:34.164501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a76e3-5156-48cb-9156-44af3ed6d950 None admin] PATH_INFO: `/v3/projects/76a7ae877c1a4d9cb04ae6eefe6d515a` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:34.165631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a76e3-5156-48cb-9156-44af3ed6d950 None admin] RBAC: Authorizing `identity:get_project(project_id=76a7ae877c1a4d9cb04ae6eefe6d515a)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:34.167060 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a76e3-5156-48cb-9156-44af3ed6d950 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:34.168690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 587/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:34:34 2022] GET /identity/v3/projects/76a7ae877c1a4d9cb04ae6eefe6d515a => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:34.174822 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d70c7-8c04-4b14-8402-8e47c6195711 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:34.277115 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d70c7-8c04-4b14-8402-8e47c6195711 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:34.309784 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec1575e-d637-43f3-8596-abbcb22cb359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:34.310380 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1575e-d637-43f3-8596-abbcb22cb359 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:34.310660 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1575e-d637-43f3-8596-abbcb22cb359 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:34.310934 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1575e-d637-43f3-8596-abbcb22cb359 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:34.311667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec1575e-d637-43f3-8596-abbcb22cb359 None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=76a7ae877c1a4d9cb04ae6eefe6d515a, user.id=37efde09728b4813b1a4a2650e1939c2) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:34:34.311958 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec1575e-d637-43f3-8596-abbcb22cb359 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:34.313435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec1575e-d637-43f3-8596-abbcb22cb359 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:34.318599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 598/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:34:34 2022] GET /identity/v3/role_assignments?user.id=37efde09728b4813b1a4a2650e1939c2&scope.project.id=76a7ae877c1a4d9cb04ae6eefe6d515a&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:36.221374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45b5c4-bdbe-4627-b37f-f4e678acec13 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:36.222184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45b5c4-bdbe-4627-b37f-f4e678acec13 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:36.222184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45b5c4-bdbe-4627-b37f-f4e678acec13 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:36.223466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 588/1186] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:36.228512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a29ba8-9046-42b8-a5ae-e0fa2afac87f None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:36.228941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a29ba8-9046-42b8-a5ae-e0fa2afac87f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:36.229086 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a29ba8-9046-42b8-a5ae-e0fa2afac87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:36.246891 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-45a29ba8-9046-42b8-a5ae-e0fa2afac87f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:36.248052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-45a29ba8-9046-42b8-a5ae-e0fa2afac87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:36.297474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:36.313677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 599/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:34:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:34:36.584633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45b5c4-bdbe-4627-b37f-f4e678acec13 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:36.684356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45b5c4-bdbe-4627-b37f-f4e678acec13 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:36.697438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:36.717717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a48d79-6454-4e03-8333-f6c918bb9bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:36.718438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a48d79-6454-4e03-8333-f6c918bb9bb6 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:36.718722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a48d79-6454-4e03-8333-f6c918bb9bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:36.718998 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a48d79-6454-4e03-8333-f6c918bb9bb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:36.729735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a48d79-6454-4e03-8333-f6c918bb9bb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-79a48d79-6454-4e03-8333-f6c918bb9bb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:34:36.735104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:34:36.736240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:34:36.738081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:34:36.738081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:34:36.738081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:34:36.738081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:34:36.738081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:34:36.738081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:34:36.738081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:34:36.738081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:34:36.738081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:34:36.738081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:34:36.738081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:34:36.738081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:34:36.738081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 589/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:34:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:34:36.743001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a29ba8-9046-42b8-a5ae-e0fa2afac87f None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:36.837289 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a29ba8-9046-42b8-a5ae-e0fa2afac87f None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:36.871428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b81478-e9de-4fe3-a939-0b82244716b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:36.872031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b81478-e9de-4fe3-a939-0b82244716b2 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:36.872312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b81478-e9de-4fe3-a939-0b82244716b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:36.872583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b81478-e9de-4fe3-a939-0b82244716b2 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:36.873275 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b81478-e9de-4fe3-a939-0b82244716b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:34:36.873579 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b81478-e9de-4fe3-a939-0b82244716b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:36.874938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b81478-e9de-4fe3-a939-0b82244716b2 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:36.882682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 600/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:34:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:36.888476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a48d79-6454-4e03-8333-f6c918bb9bb6 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:36.986144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a48d79-6454-4e03-8333-f6c918bb9bb6 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:37.019532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed0040-f8d4-40bf-b96f-9fa8b78d0189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:37.020131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed0040-f8d4-40bf-b96f-9fa8b78d0189 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:37.020407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed0040-f8d4-40bf-b96f-9fa8b78d0189 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:37.020700 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed0040-f8d4-40bf-b96f-9fa8b78d0189 None admin] PATH_INFO: `/v3/users/37efde09728b4813b1a4a2650e1939c2` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:37.021789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed0040-f8d4-40bf-b96f-9fa8b78d0189 None admin] RBAC: Authorizing `identity:get_user(user_id=37efde09728b4813b1a4a2650e1939c2)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:37.023198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed0040-f8d4-40bf-b96f-9fa8b78d0189 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:37.025206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 590/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:34:36 2022] GET /identity/v3/users/37efde09728b4813b1a4a2650e1939c2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:37.030868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b81478-e9de-4fe3-a939-0b82244716b2 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:37.077721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:37.125909 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b81478-e9de-4fe3-a939-0b82244716b2 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:37.157855 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba52710-dc67-4c48-99a5-ec0f9514925f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:37.158466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba52710-dc67-4c48-99a5-ec0f9514925f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:37.159376 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba52710-dc67-4c48-99a5-ec0f9514925f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:37.159903 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba52710-dc67-4c48-99a5-ec0f9514925f None admin] PATH_INFO: `/v3/projects/76a7ae877c1a4d9cb04ae6eefe6d515a` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:37.161617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba52710-dc67-4c48-99a5-ec0f9514925f None admin] RBAC: Authorizing `identity:get_project(project_id=76a7ae877c1a4d9cb04ae6eefe6d515a)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:37.163028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba52710-dc67-4c48-99a5-ec0f9514925f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:37.164631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 601/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:34:37 2022] GET /identity/v3/projects/76a7ae877c1a4d9cb04ae6eefe6d515a => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:37.170633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed0040-f8d4-40bf-b96f-9fa8b78d0189 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:37.211729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:37.267354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed0040-f8d4-40bf-b96f-9fa8b78d0189 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:37.300952 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242b6c6-829c-4c6a-af1f-63345d94d267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:37.301609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242b6c6-829c-4c6a-af1f-63345d94d267 None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:37.301892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242b6c6-829c-4c6a-af1f-63345d94d267 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:37.302177 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242b6c6-829c-4c6a-af1f-63345d94d267 None admin] PATH_INFO: `/v3/projects/76a7ae877c1a4d9cb04ae6eefe6d515a/users/37efde09728b4813b1a4a2650e1939c2/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:37.304056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242b6c6-829c-4c6a-af1f-63345d94d267 None admin] RBAC: Authorizing `identity:create_grant(project_id=76a7ae877c1a4d9cb04ae6eefe6d515a, user_id=37efde09728b4813b1a4a2650e1939c2, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:37.305507 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242b6c6-829c-4c6a-af1f-63345d94d267 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:37.405312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 591/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:34:37 2022] PUT /identity/v3/projects/76a7ae877c1a4d9cb04ae6eefe6d515a/users/37efde09728b4813b1a4a2650e1939c2/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:34:39.297425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e7fe0-f08b-41e1-94d7-6478ff5a50cf None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:39.297964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e7fe0-f08b-41e1-94d7-6478ff5a50cf None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:39.297964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e7fe0-f08b-41e1-94d7-6478ff5a50cf None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:39.299328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 602/1193] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:34:39.304667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d18e0-74d8-4e3b-b8c4-6a6f2e7d07b0 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:39.305086 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d18e0-74d8-4e3b-b8c4-6a6f2e7d07b0 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:39.305229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d18e0-74d8-4e3b-b8c4-6a6f2e7d07b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:39.322728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-8d9d18e0-74d8-4e3b-b8c4-6a6f2e7d07b0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:34:39.323904 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-8d9d18e0-74d8-4e3b-b8c4-6a6f2e7d07b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:34:39.388035 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 592/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:34:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:34:39.657466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e7fe0-f08b-41e1-94d7-6478ff5a50cf None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:39.758279 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e7fe0-f08b-41e1-94d7-6478ff5a50cf None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:39.773926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:39.791298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a22705-3e01-4437-a27d-d744b4d6c9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:39.791917 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a22705-3e01-4437-a27d-d744b4d6c9ae None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:39.792201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a22705-3e01-4437-a27d-d744b4d6c9ae None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:39.792479 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a22705-3e01-4437-a27d-d744b4d6c9ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:39.802795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a22705-3e01-4437-a27d-d744b4d6c9ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-53a22705-3e01-4437-a27d-d744b4d6c9ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:34:39.807542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:34:39.808624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:34:39.809550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:34:39.809550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:34:39.809550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:34:39.809550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:34:39.809550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:34:39.809550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:34:39.809550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:34:39.809550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:34:39.809550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:34:39.809550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:34:39.809550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:34:39.809550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:34:39.809550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 603/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:34:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:34:39.814597 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d18e0-74d8-4e3b-b8c4-6a6f2e7d07b0 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:39.842892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:39.909792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d18e0-74d8-4e3b-b8c4-6a6f2e7d07b0 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:39.943242 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec095a18-7b26-45cb-bae2-9b07b82e9d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:39.943833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec095a18-7b26-45cb-bae2-9b07b82e9d72 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:39.944113 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec095a18-7b26-45cb-bae2-9b07b82e9d72 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:39.944386 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec095a18-7b26-45cb-bae2-9b07b82e9d72 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:39.945066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec095a18-7b26-45cb-bae2-9b07b82e9d72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:34:39.945371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec095a18-7b26-45cb-bae2-9b07b82e9d72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:39.946730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec095a18-7b26-45cb-bae2-9b07b82e9d72 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:39.954394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 593/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:34:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:39.960199 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a22705-3e01-4437-a27d-d744b4d6c9ae None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:40.058864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a22705-3e01-4437-a27d-d744b4d6c9ae None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:40.091845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910da936-9e42-439f-ad01-063186764bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:40.092463 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910da936-9e42-439f-ad01-063186764bce None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:40.092785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910da936-9e42-439f-ad01-063186764bce None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:40.093066 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910da936-9e42-439f-ad01-063186764bce None admin] PATH_INFO: `/v3/users/37efde09728b4813b1a4a2650e1939c2` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:40.094190 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910da936-9e42-439f-ad01-063186764bce None admin] RBAC: Authorizing `identity:get_user(user_id=37efde09728b4813b1a4a2650e1939c2)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:40.095613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910da936-9e42-439f-ad01-063186764bce None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:40.097979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 604/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 24 07:34:39 2022] GET /identity/v3/users/37efde09728b4813b1a4a2650e1939c2 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:40.103768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec095a18-7b26-45cb-bae2-9b07b82e9d72 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:40.192008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:40.200315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec095a18-7b26-45cb-bae2-9b07b82e9d72 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:40.235503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a25c7-56c8-484d-a1d7-cc89e714bfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:40.236096 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a25c7-56c8-484d-a1d7-cc89e714bfd7 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:40.236379 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a25c7-56c8-484d-a1d7-cc89e714bfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:40.236703 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a25c7-56c8-484d-a1d7-cc89e714bfd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:40.301400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a25c7-56c8-484d-a1d7-cc89e714bfd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:40.302835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a25c7-56c8-484d-a1d7-cc89e714bfd7 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:40.388488 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:40.397586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 594/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:34:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:34:40.403665 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910da936-9e42-439f-ad01-063186764bce None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:40.448972 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:40.501060 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910da936-9e42-439f-ad01-063186764bce None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:40.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b8f67-f70b-4ca3-bb87-9a61e2c2eb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:40.534492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b8f67-f70b-4ca3-bb87-9a61e2c2eb93 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:40.534926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b8f67-f70b-4ca3-bb87-9a61e2c2eb93 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:40.535204 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b8f67-f70b-4ca3-bb87-9a61e2c2eb93 None admin] PATH_INFO: `/v3/projects/76a7ae877c1a4d9cb04ae6eefe6d515a` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:40.536305 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b8f67-f70b-4ca3-bb87-9a61e2c2eb93 None admin] RBAC: Authorizing `identity:get_project(project_id=76a7ae877c1a4d9cb04ae6eefe6d515a)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:40.537759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b8f67-f70b-4ca3-bb87-9a61e2c2eb93 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:40.539363 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 605/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 24 07:34:40 2022] GET /identity/v3/projects/76a7ae877c1a4d9cb04ae6eefe6d515a => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:40.545389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a25c7-56c8-484d-a1d7-cc89e714bfd7 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:34:40.644607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a25c7-56c8-484d-a1d7-cc89e714bfd7 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:34:40.677243 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77058f0-2f3e-4ac9-adb8-d27e586dd8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:34:40.677825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77058f0-2f3e-4ac9-adb8-d27e586dd8a3 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:40.678101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77058f0-2f3e-4ac9-adb8-d27e586dd8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:40.678366 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77058f0-2f3e-4ac9-adb8-d27e586dd8a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:40.679066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77058f0-2f3e-4ac9-adb8-d27e586dd8a3 None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=76a7ae877c1a4d9cb04ae6eefe6d515a, user.id=37efde09728b4813b1a4a2650e1939c2) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:34:40.679372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77058f0-2f3e-4ac9-adb8-d27e586dd8a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:34:40.680804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77058f0-2f3e-4ac9-adb8-d27e586dd8a3 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:34:40.686232 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 595/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:34:40 2022] GET /identity/v3/role_assignments?user.id=37efde09728b4813b1a4a2650e1939c2&scope.project.id=76a7ae877c1a4d9cb04ae6eefe6d515a&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:34:50.529005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=44 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:34:50.683731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=72 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:51.619657 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba502f-4df3-499e-8437-e7b572f3ee44 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:51.620238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba502f-4df3-499e-8437-e7b572f3ee44 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:51.620238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba502f-4df3-499e-8437-e7b572f3ee44 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:51.621652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 606/1201] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:51.626992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50426a32-16c0-4170-a36e-bdc0159cc2e4 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:51.627393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50426a32-16c0-4170-a36e-bdc0159cc2e4 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:51.627541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50426a32-16c0-4170-a36e-bdc0159cc2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:51.645224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-50426a32-16c0-4170-a36e-bdc0159cc2e4 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:51.646404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-50426a32-16c0-4170-a36e-bdc0159cc2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:51.712619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 596/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:35:51.987278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba502f-4df3-499e-8437-e7b572f3ee44 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:52.090750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba502f-4df3-499e-8437-e7b572f3ee44 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:52.124439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e048ec3-080b-4a84-b057-8bfcb9b5004c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:52.125082 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e048ec3-080b-4a84-b057-8bfcb9b5004c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:52.125363 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e048ec3-080b-4a84-b057-8bfcb9b5004c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:52.125658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e048ec3-080b-4a84-b057-8bfcb9b5004c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:52.135479 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e048ec3-080b-4a84-b057-8bfcb9b5004c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:52.138430 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e048ec3-080b-4a84-b057-8bfcb9b5004c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-1e048ec3-080b-4a84-b057-8bfcb9b5004c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:35:52.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:52.152150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:35:52.153228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:35:52.153228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:35:52.153228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:35:52.153228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 607/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:52.158655 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50426a32-16c0-4170-a36e-bdc0159cc2e4 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:52.258855 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50426a32-16c0-4170-a36e-bdc0159cc2e4 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:52.287111 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:52.292884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d4885-9c6b-4deb-8a1a-87249abff06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:52.293693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d4885-9c6b-4deb-8a1a-87249abff06d None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:52.293804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d4885-9c6b-4deb-8a1a-87249abff06d None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:52.294048 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d4885-9c6b-4deb-8a1a-87249abff06d None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:52.294685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d4885-9c6b-4deb-8a1a-87249abff06d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:52.294992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d4885-9c6b-4deb-8a1a-87249abff06d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:52.296353 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d4885-9c6b-4deb-8a1a-87249abff06d None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:52.307913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 597/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:52.314274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e048ec3-080b-4a84-b057-8bfcb9b5004c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:52.367050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:52.412525 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e048ec3-080b-4a84-b057-8bfcb9b5004c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:52.448202 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b4981-5661-4c74-b799-303d38d73c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:52.448828 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b4981-5661-4c74-b799-303d38d73c06 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:52.449106 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b4981-5661-4c74-b799-303d38d73c06 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:52.449374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b4981-5661-4c74-b799-303d38d73c06 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:52.450014 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b4981-5661-4c74-b799-303d38d73c06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:52.451426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b4981-5661-4c74-b799-303d38d73c06 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:52.531754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 608/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 24 07:35:52 2022] POST /identity/v3/users => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:35:54.463905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d544f67-4b3a-4418-8d10-8eb8e4d0f48c None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:54.464459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d544f67-4b3a-4418-8d10-8eb8e4d0f48c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:54.464459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d544f67-4b3a-4418-8d10-8eb8e4d0f48c None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:54.465848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 598/1206] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:54.472060 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8211c-d736-4350-9cfb-a8a6facfc0e6 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:54.472453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8211c-d736-4350-9cfb-a8a6facfc0e6 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:54.472608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8211c-d736-4350-9cfb-a8a6facfc0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:54.491317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-dff8211c-d736-4350-9cfb-a8a6facfc0e6 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:54.492494 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-dff8211c-d736-4350-9cfb-a8a6facfc0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:54.559701 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 609/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:35:54.826757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d544f67-4b3a-4418-8d10-8eb8e4d0f48c None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:54.926105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d544f67-4b3a-4418-8d10-8eb8e4d0f48c None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:54.958778 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303aa42-b698-4872-a93d-c6e4b83ac3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:54.959393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303aa42-b698-4872-a93d-c6e4b83ac3e9 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:54.959670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303aa42-b698-4872-a93d-c6e4b83ac3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:54.959939 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303aa42-b698-4872-a93d-c6e4b83ac3e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:54.970222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303aa42-b698-4872-a93d-c6e4b83ac3e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-f303aa42-b698-4872-a93d-c6e4b83ac3e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:54.975105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:54.976210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:54.977272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:54.977272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:54.977272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:54.977272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:54.977272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:54.977272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:54.977272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:54.977272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:54.977272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:54.977272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:54.977272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:35:54.977272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:35:54.977272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 599/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:35:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:54.982338 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8211c-d736-4350-9cfb-a8a6facfc0e6 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:55.022326 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:55.080527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8211c-d736-4350-9cfb-a8a6facfc0e6 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:55.113809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae530722-30ca-433d-8c70-6b0e07056542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:55.114426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae530722-30ca-433d-8c70-6b0e07056542 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.114704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae530722-30ca-433d-8c70-6b0e07056542 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.114973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae530722-30ca-433d-8c70-6b0e07056542 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.115709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae530722-30ca-433d-8c70-6b0e07056542 None admin] RBAC: Adding query filter params (name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:55.116008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae530722-30ca-433d-8c70-6b0e07056542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:55.117420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae530722-30ca-433d-8c70-6b0e07056542 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:55.125355 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 610/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:35:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:55.131665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303aa42-b698-4872-a93d-c6e4b83ac3e9 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:55.181537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:55.230299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303aa42-b698-4872-a93d-c6e4b83ac3e9 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:55.263508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80510bb0-5c67-4e65-9c67-7e898f664a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:55.264120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80510bb0-5c67-4e65-9c67-7e898f664a32 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.264399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80510bb0-5c67-4e65-9c67-7e898f664a32 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.264693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80510bb0-5c67-4e65-9c67-7e898f664a32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.275423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80510bb0-5c67-4e65-9c67-7e898f664a32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:55.276955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80510bb0-5c67-4e65-9c67-7e898f664a32 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-80510bb0-5c67-4e65-9c67-7e898f664a32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:35:55.291390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:55.292523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:35:55.293879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:35:55.293879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:35:55.293879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:35:55.293879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 600/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:55.298541 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae530722-30ca-433d-8c70-6b0e07056542 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:55.395409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:55.398149 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae530722-30ca-433d-8c70-6b0e07056542 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:55.430883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53eb37-c2d2-4f69-9b8f-6cd01c243f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:55.431507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53eb37-c2d2-4f69-9b8f-6cd01c243f29 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.431794 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53eb37-c2d2-4f69-9b8f-6cd01c243f29 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.432077 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53eb37-c2d2-4f69-9b8f-6cd01c243f29 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.432734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53eb37-c2d2-4f69-9b8f-6cd01c243f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:55.433038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53eb37-c2d2-4f69-9b8f-6cd01c243f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:55.434410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53eb37-c2d2-4f69-9b8f-6cd01c243f29 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:55.445851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 611/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:55.451589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80510bb0-5c67-4e65-9c67-7e898f664a32 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:55.553246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80510bb0-5c67-4e65-9c67-7e898f664a32 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:55.554756 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:55.587229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03f715-c06f-4b51-ada5-c2c379331d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:55.587845 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03f715-c06f-4b51-ada5-c2c379331d6e None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.588138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03f715-c06f-4b51-ada5-c2c379331d6e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.588392 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03f715-c06f-4b51-ada5-c2c379331d6e None admin] PATH_INFO: `/v3/users/placement` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.595985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03f715-c06f-4b51-ada5-c2c379331d6e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:55.597728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03f715-c06f-4b51-ada5-c2c379331d6e None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-2b03f715-c06f-4b51-ada5-c2c379331d6e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:35:55.609170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:55.616683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:55.617956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:55.617956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:55.617956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:35:55.617956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:35:55.617956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:35:55.617956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:35:55.617956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:35:55.617956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:35:55.617956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:35:55.617956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:35:55.617956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 601/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:55.618733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53eb37-c2d2-4f69-9b8f-6cd01c243f29 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:55.713359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53eb37-c2d2-4f69-9b8f-6cd01c243f29 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:55.746059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b821a-2445-44d0-a430-bb86806dfff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:55.746660 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b821a-2445-44d0-a430-bb86806dfff7 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.746942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b821a-2445-44d0-a430-bb86806dfff7 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.747214 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b821a-2445-44d0-a430-bb86806dfff7 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.747927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b821a-2445-44d0-a430-bb86806dfff7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:55.748232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b821a-2445-44d0-a430-bb86806dfff7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:55.749678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b821a-2445-44d0-a430-bb86806dfff7 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:55.766054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 612/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Aug 24 07:35:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:55.772532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03f715-c06f-4b51-ada5-c2c379331d6e None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:55.871324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03f715-c06f-4b51-ada5-c2c379331d6e None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:55.905242 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e409f-1a31-4fbe-b0b7-923272b6cb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:55.905836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e409f-1a31-4fbe-b0b7-923272b6cb5c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.906113 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e409f-1a31-4fbe-b0b7-923272b6cb5c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.906385 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e409f-1a31-4fbe-b0b7-923272b6cb5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.934859 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:55.972114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e409f-1a31-4fbe-b0b7-923272b6cb5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:55.973537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e409f-1a31-4fbe-b0b7-923272b6cb5c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:56.069964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 602/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:35:56.075971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b821a-2445-44d0-a430-bb86806dfff7 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:56.095375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:56.173679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b821a-2445-44d0-a430-bb86806dfff7 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:56.207169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42679a-4101-4f99-b0d7-397ff0cdb242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:56.207789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42679a-4101-4f99-b0d7-397ff0cdb242 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:56.208071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42679a-4101-4f99-b0d7-397ff0cdb242 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:56.208339 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42679a-4101-4f99-b0d7-397ff0cdb242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:56.217987 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42679a-4101-4f99-b0d7-397ff0cdb242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:56.219410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42679a-4101-4f99-b0d7-397ff0cdb242 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-9d42679a-4101-4f99-b0d7-397ff0cdb242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:35:56.233961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:56.235673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:35:56.237132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:35:56.237132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:35:56.237132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:35:56.237132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 613/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:56.241215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e409f-1a31-4fbe-b0b7-923272b6cb5c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:56.303237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:56.340993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e409f-1a31-4fbe-b0b7-923272b6cb5c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:56.374572 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5e368-cf3e-44d9-abb5-5d6c2c0a16ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:56.379675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5e368-cf3e-44d9-abb5-5d6c2c0a16ff None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:56.379675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5e368-cf3e-44d9-abb5-5d6c2c0a16ff None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:56.379675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5e368-cf3e-44d9-abb5-5d6c2c0a16ff None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:56.379675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5e368-cf3e-44d9-abb5-5d6c2c0a16ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:56.379675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5e368-cf3e-44d9-abb5-5d6c2c0a16ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:56.379675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5e368-cf3e-44d9-abb5-5d6c2c0a16ff None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:56.389492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 603/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:56.395983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42679a-4101-4f99-b0d7-397ff0cdb242 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:56.464759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:56.494423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42679a-4101-4f99-b0d7-397ff0cdb242 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:56.526647 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a63d21-37b7-428c-aa28-154a90b7f108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:56.527276 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a63d21-37b7-428c-aa28-154a90b7f108 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:56.527559 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a63d21-37b7-428c-aa28-154a90b7f108 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:56.527850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a63d21-37b7-428c-aa28-154a90b7f108 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:56.535193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a63d21-37b7-428c-aa28-154a90b7f108 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:56.537438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a63d21-37b7-428c-aa28-154a90b7f108 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-09a63d21-37b7-428c-aa28-154a90b7f108 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:35:56.549415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:56.550521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:56.551851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:35:56.551851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:35:56.551851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:35:56.551851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:35:56.551851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:35:56.551851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:35:56.551851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:35:56.551851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:35:56.551851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 614/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:35:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:56.556356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5e368-cf3e-44d9-abb5-5d6c2c0a16ff None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:56.656396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5e368-cf3e-44d9-abb5-5d6c2c0a16ff None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:56.678612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:56.690300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6eecd-6156-40ac-a651-ac3041542e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:56.690895 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6eecd-6156-40ac-a651-ac3041542e9b None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:56.691178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6eecd-6156-40ac-a651-ac3041542e9b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:56.691456 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6eecd-6156-40ac-a651-ac3041542e9b None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:56.692140 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6eecd-6156-40ac-a651-ac3041542e9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:56.692449 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6eecd-6156-40ac-a651-ac3041542e9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:56.693902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6eecd-6156-40ac-a651-ac3041542e9b None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:56.706010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 604/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:35:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:56.712478 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a63d21-37b7-428c-aa28-154a90b7f108 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:56.814851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a63d21-37b7-428c-aa28-154a90b7f108 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:56.846688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:56.852255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ded697-e90e-44c3-b766-81d8a7363f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:56.852888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded697-e90e-44c3-b766-81d8a7363f3d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:56.853173 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded697-e90e-44c3-b766-81d8a7363f3d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:56.853448 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded697-e90e-44c3-b766-81d8a7363f3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:56.854167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ded697-e90e-44c3-b766-81d8a7363f3d None admin] RBAC: Adding query filter params (role.id=3acdf08298c84325b53a0e286cc53b30, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=ea3816d5ae054741ac2c5a2eec3eb617) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:56.854474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ded697-e90e-44c3-b766-81d8a7363f3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:56.855893 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ded697-e90e-44c3-b766-81d8a7363f3d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:56.861264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 615/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:56 2022] GET /identity/v3/role_assignments?user.id=ea3816d5ae054741ac2c5a2eec3eb617&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=3acdf08298c84325b53a0e286cc53b30 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:58.728830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe89557-bfcc-490f-831d-d8834a0d2c0d None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:58.729356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe89557-bfcc-490f-831d-d8834a0d2c0d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:58.729448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe89557-bfcc-490f-831d-d8834a0d2c0d None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:58.730749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 605/1220] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:58.736444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cf34c-9216-43af-a0b2-7f4795331dbb None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:58.737823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cf34c-9216-43af-a0b2-7f4795331dbb None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:58.737823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cf34c-9216-43af-a0b2-7f4795331dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:58.755872 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-195cf34c-9216-43af-a0b2-7f4795331dbb None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:58.757212 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-195cf34c-9216-43af-a0b2-7f4795331dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:58.823189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 616/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:35:59.098619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe89557-bfcc-490f-831d-d8834a0d2c0d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:59.198417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe89557-bfcc-490f-831d-d8834a0d2c0d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:59.232438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19ea1b-90bf-4b84-80b0-9de7e562b4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:59.233079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19ea1b-90bf-4b84-80b0-9de7e562b4a7 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:59.233360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19ea1b-90bf-4b84-80b0-9de7e562b4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:59.233632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19ea1b-90bf-4b84-80b0-9de7e562b4a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:59.243753 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19ea1b-90bf-4b84-80b0-9de7e562b4a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-3a19ea1b-90bf-4b84-80b0-9de7e562b4a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:59.248377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:59.249460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:59.250954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:59.250954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:59.250954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:59.250954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:59.250954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:59.250954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:59.250954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:59.250954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:59.250954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:59.250954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:59.250954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:35:59.250954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:35:59.250954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 606/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:35:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:59.255530 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cf34c-9216-43af-a0b2-7f4795331dbb None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:59.352940 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cf34c-9216-43af-a0b2-7f4795331dbb None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:59.375269 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:59.385540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c9e2c-5fb7-4869-ac23-3a71579f3804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:59.386162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c9e2c-5fb7-4869-ac23-3a71579f3804 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:59.386447 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c9e2c-5fb7-4869-ac23-3a71579f3804 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:59.386724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c9e2c-5fb7-4869-ac23-3a71579f3804 None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:59.387387 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c9e2c-5fb7-4869-ac23-3a71579f3804 None admin] RBAC: Adding query filter params (name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:59.387693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c9e2c-5fb7-4869-ac23-3a71579f3804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:59.389093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c9e2c-5fb7-4869-ac23-3a71579f3804 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:59.397214 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 617/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:35:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:59.402966 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19ea1b-90bf-4b84-80b0-9de7e562b4a7 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:59.445672 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:59.499427 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19ea1b-90bf-4b84-80b0-9de7e562b4a7 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:59.533351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70d8a6-06e3-4628-bd4d-f97498d98802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:59.533982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70d8a6-06e3-4628-bd4d-f97498d98802 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:59.534262 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70d8a6-06e3-4628-bd4d-f97498d98802 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:59.534535 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70d8a6-06e3-4628-bd4d-f97498d98802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:59.544314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70d8a6-06e3-4628-bd4d-f97498d98802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:59.545809 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70d8a6-06e3-4628-bd4d-f97498d98802 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-3b70d8a6-06e3-4628-bd4d-f97498d98802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:35:59.559758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:59.560944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:35:59.562036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:35:59.562036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:35:59.562036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:35:59.562036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 607/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:59.567387 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c9e2c-5fb7-4869-ac23-3a71579f3804 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:59.682771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c9e2c-5fb7-4869-ac23-3a71579f3804 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:59.716377 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6baef-0351-4fbd-9e9f-68c2b86f0f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:59.717092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6baef-0351-4fbd-9e9f-68c2b86f0f12 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:59.717377 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6baef-0351-4fbd-9e9f-68c2b86f0f12 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:59.717646 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6baef-0351-4fbd-9e9f-68c2b86f0f12 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:59.718275 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6baef-0351-4fbd-9e9f-68c2b86f0f12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:59.718572 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6baef-0351-4fbd-9e9f-68c2b86f0f12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:59.720286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6baef-0351-4fbd-9e9f-68c2b86f0f12 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:59.731890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 618/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:59.737630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70d8a6-06e3-4628-bd4d-f97498d98802 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:59.830196 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:59.836577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70d8a6-06e3-4628-bd4d-f97498d98802 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:59.869482 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad47b33-1b37-481f-a3d8-3a1023cc8dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:59.870109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad47b33-1b37-481f-a3d8-3a1023cc8dda None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:59.870390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad47b33-1b37-481f-a3d8-3a1023cc8dda None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:59.870664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad47b33-1b37-481f-a3d8-3a1023cc8dda None admin] PATH_INFO: `/v3/users/placement` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:59.878072 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad47b33-1b37-481f-a3d8-3a1023cc8dda None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:59.879506 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad47b33-1b37-481f-a3d8-3a1023cc8dda None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-2ad47b33-1b37-481f-a3d8-3a1023cc8dda None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:35:59.890827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:59.893015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:59.894045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:59.894045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:59.894045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:35:59.894045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:35:59.894045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:35:59.894045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:35:59.894045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:35:59.894045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:35:59.894045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:35:59.894045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:35:59.894045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 608/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:59.898399 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6baef-0351-4fbd-9e9f-68c2b86f0f12 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:59.945759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:59.997534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6baef-0351-4fbd-9e9f-68c2b86f0f12 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:00.031498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ceb33-7240-4361-8054-859630ae6e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:00.032134 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ceb33-7240-4361-8054-859630ae6e06 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.032556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ceb33-7240-4361-8054-859630ae6e06 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.032887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ceb33-7240-4361-8054-859630ae6e06 None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.033617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ceb33-7240-4361-8054-859630ae6e06 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:00.033929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ceb33-7240-4361-8054-859630ae6e06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:00.035447 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ceb33-7240-4361-8054-859630ae6e06 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:00.053169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 619/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Aug 24 07:35:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:00.058942 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad47b33-1b37-481f-a3d8-3a1023cc8dda None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:00.158693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad47b33-1b37-481f-a3d8-3a1023cc8dda None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:00.190692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ffc5a-36ed-4751-9144-5dfa7ffceaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:00.191305 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ffc5a-36ed-4751-9144-5dfa7ffceaea None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.191582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ffc5a-36ed-4751-9144-5dfa7ffceaea None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.191853 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ffc5a-36ed-4751-9144-5dfa7ffceaea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.202046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ffc5a-36ed-4751-9144-5dfa7ffceaea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:00.203417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ffc5a-36ed-4751-9144-5dfa7ffceaea None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-443ffc5a-36ed-4751-9144-5dfa7ffceaea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:00.218150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:00.221636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:00.222548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:00.222548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:00.222548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:00.222548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 609/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:00.226223 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ceb33-7240-4361-8054-859630ae6e06 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:00.323614 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ceb33-7240-4361-8054-859630ae6e06 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:00.325951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:00.356343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9e1cb-263b-4256-b83e-f3d0da4450d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:00.356964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9e1cb-263b-4256-b83e-f3d0da4450d1 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.357264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9e1cb-263b-4256-b83e-f3d0da4450d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.357520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9e1cb-263b-4256-b83e-f3d0da4450d1 None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.358161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9e1cb-263b-4256-b83e-f3d0da4450d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:00.358460 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9e1cb-263b-4256-b83e-f3d0da4450d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:00.359819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9e1cb-263b-4256-b83e-f3d0da4450d1 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:00.370823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 620/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:00.377181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ffc5a-36ed-4751-9144-5dfa7ffceaea None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:00.386434 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:00.476762 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ffc5a-36ed-4751-9144-5dfa7ffceaea None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:00.509327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba97bb4-95bf-4b3d-b468-06bb91c6c14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:00.509942 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba97bb4-95bf-4b3d-b468-06bb91c6c14c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.510221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba97bb4-95bf-4b3d-b468-06bb91c6c14c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.510509 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba97bb4-95bf-4b3d-b468-06bb91c6c14c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.517476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba97bb4-95bf-4b3d-b468-06bb91c6c14c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:00.518931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba97bb4-95bf-4b3d-b468-06bb91c6c14c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-cba97bb4-95bf-4b3d-b468-06bb91c6c14c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:00.529766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:00.533178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:00.534836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:00.534836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:00.534836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:00.534836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:00.534836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:00.534836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:00.534836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:00.534836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:00.534836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 610/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:00.536960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9e1cb-263b-4256-b83e-f3d0da4450d1 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:00.637011 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9e1cb-263b-4256-b83e-f3d0da4450d1 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:00.669759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a543e5-cf24-4ba4-b997-41f2c08206da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:00.670355 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a543e5-cf24-4ba4-b997-41f2c08206da None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.670634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a543e5-cf24-4ba4-b997-41f2c08206da None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.670907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a543e5-cf24-4ba4-b997-41f2c08206da None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.671584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a543e5-cf24-4ba4-b997-41f2c08206da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:00.671891 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a543e5-cf24-4ba4-b997-41f2c08206da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:00.673844 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a543e5-cf24-4ba4-b997-41f2c08206da None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:00.689046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 621/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:00.695173 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba97bb4-95bf-4b3d-b468-06bb91c6c14c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:00.760504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:00.793627 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba97bb4-95bf-4b3d-b468-06bb91c6c14c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:00.828179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1eb77-3d2e-4d3a-ae05-ee83fdfe370a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:00.828893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1eb77-3d2e-4d3a-ae05-ee83fdfe370a None admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.829240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1eb77-3d2e-4d3a-ae05-ee83fdfe370a None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.829532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1eb77-3d2e-4d3a-ae05-ee83fdfe370a None admin] PATH_INFO: `/v3/projects/fc060764023e44a7ade80eeed98a235f/users/ea3816d5ae054741ac2c5a2eec3eb617/roles/3acdf08298c84325b53a0e286cc53b30` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.844212 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1eb77-3d2e-4d3a-ae05-ee83fdfe370a None admin] RBAC: Authorizing `identity:create_grant(project_id=fc060764023e44a7ade80eeed98a235f, user_id=ea3816d5ae054741ac2c5a2eec3eb617, role_id=3acdf08298c84325b53a0e286cc53b30)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:00.845886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1eb77-3d2e-4d3a-ae05-ee83fdfe370a None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:00.948514 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 611/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:36:00 2022] PUT /identity/v3/projects/fc060764023e44a7ade80eeed98a235f/users/ea3816d5ae054741ac2c5a2eec3eb617/roles/3acdf08298c84325b53a0e286cc53b30 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:02.894229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9fea7-864f-4a84-9af3-394433d74186 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:02.894863 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9fea7-864f-4a84-9af3-394433d74186 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:02.894863 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9fea7-864f-4a84-9af3-394433d74186 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:02.896130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 622/1233] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:02.901599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9147886-2ad9-4be9-abdb-45bff33ffd8a None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:02.913549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9147886-2ad9-4be9-abdb-45bff33ffd8a None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:02.913549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9147886-2ad9-4be9-abdb-45bff33ffd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:02.919803 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-f9147886-2ad9-4be9-abdb-45bff33ffd8a None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:02.921022 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f9147886-2ad9-4be9-abdb-45bff33ffd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:02.987768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 612/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:03.251537 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9fea7-864f-4a84-9af3-394433d74186 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:03.276869 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:03.351089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9fea7-864f-4a84-9af3-394433d74186 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:03.383236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c741849-4a31-43fd-9943-e9471b4a4c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:03.387732 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c741849-4a31-43fd-9943-e9471b4a4c62 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:03.387732 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c741849-4a31-43fd-9943-e9471b4a4c62 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:03.387732 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c741849-4a31-43fd-9943-e9471b4a4c62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:03.394677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c741849-4a31-43fd-9943-e9471b4a4c62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-6c741849-4a31-43fd-9943-e9471b4a4c62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:03.399288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:03.400506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:03.401817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:03.401817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:03.401817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:03.401817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:03.401817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:03.401817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:03.401817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:03.401817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:03.401817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:03.401817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:03.401817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:03.401817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:36:03.401817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 623/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:03.406494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9147886-2ad9-4be9-abdb-45bff33ffd8a None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:03.452210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:03.504303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9147886-2ad9-4be9-abdb-45bff33ffd8a None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:03.539239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94edee-5340-49ac-8e0b-790076958a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:03.539852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94edee-5340-49ac-8e0b-790076958a55 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:03.543933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94edee-5340-49ac-8e0b-790076958a55 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:03.543933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94edee-5340-49ac-8e0b-790076958a55 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:03.543933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94edee-5340-49ac-8e0b-790076958a55 None admin] RBAC: Adding query filter params (name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:03.543933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94edee-5340-49ac-8e0b-790076958a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:03.543933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94edee-5340-49ac-8e0b-790076958a55 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:03.550515 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 613/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:03.556317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c741849-4a31-43fd-9943-e9471b4a4c62 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:03.642256 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:03.657645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c741849-4a31-43fd-9943-e9471b4a4c62 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:03.691783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58ab49-d619-4b82-9069-8652bc61afba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:03.692408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58ab49-d619-4b82-9069-8652bc61afba None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:03.692713 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58ab49-d619-4b82-9069-8652bc61afba None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:03.692988 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58ab49-d619-4b82-9069-8652bc61afba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:03.703445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58ab49-d619-4b82-9069-8652bc61afba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:03.704992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58ab49-d619-4b82-9069-8652bc61afba None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-1d58ab49-d619-4b82-9069-8652bc61afba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:03.718519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:03.723091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:03.724242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:03.724242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:03.724242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:36:03.724242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 624/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:03.726062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94edee-5340-49ac-8e0b-790076958a55 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:03.826103 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94edee-5340-49ac-8e0b-790076958a55 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:03.835725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:03.860600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca694819-b5a9-43b1-9939-b8e833fcd379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:03.861213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca694819-b5a9-43b1-9939-b8e833fcd379 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:03.861494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca694819-b5a9-43b1-9939-b8e833fcd379 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:03.861768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca694819-b5a9-43b1-9939-b8e833fcd379 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:03.862398 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca694819-b5a9-43b1-9939-b8e833fcd379 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:03.862701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca694819-b5a9-43b1-9939-b8e833fcd379 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:03.864067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca694819-b5a9-43b1-9939-b8e833fcd379 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:03.875351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 614/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:03.881862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58ab49-d619-4b82-9069-8652bc61afba None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:03.984243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58ab49-d619-4b82-9069-8652bc61afba None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:04.018118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b823f9c-c1c8-4212-924d-f6cedb057a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:04.018752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b823f9c-c1c8-4212-924d-f6cedb057a2f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:04.019035 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b823f9c-c1c8-4212-924d-f6cedb057a2f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:04.019312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b823f9c-c1c8-4212-924d-f6cedb057a2f None admin] PATH_INFO: `/v3/users/placement` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:04.021963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:04.026888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b823f9c-c1c8-4212-924d-f6cedb057a2f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:04.028346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b823f9c-c1c8-4212-924d-f6cedb057a2f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-7b823f9c-c1c8-4212-924d-f6cedb057a2f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:04.039337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:04.042558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:04.043735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:04.043735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:04.043735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:04.043735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:04.043735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:04.043735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:04.043735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:04.043735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:04.043735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:36:04.043735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:36:04.043735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 625/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:04.046251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca694819-b5a9-43b1-9939-b8e833fcd379 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:04.146204 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca694819-b5a9-43b1-9939-b8e833fcd379 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:04.180231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134f5d4-e520-4b1f-a659-cb9ca651c1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:04.180872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134f5d4-e520-4b1f-a659-cb9ca651c1ed None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:04.181148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134f5d4-e520-4b1f-a659-cb9ca651c1ed None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:04.181449 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134f5d4-e520-4b1f-a659-cb9ca651c1ed None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:04.182167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134f5d4-e520-4b1f-a659-cb9ca651c1ed None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:04.182476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134f5d4-e520-4b1f-a659-cb9ca651c1ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:04.183922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134f5d4-e520-4b1f-a659-cb9ca651c1ed None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:04.200967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 615/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Aug 24 07:36:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:04.206622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b823f9c-c1c8-4212-924d-f6cedb057a2f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:04.308359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b823f9c-c1c8-4212-924d-f6cedb057a2f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:04.340909 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09847fdd-88dd-4b28-81b1-0698c3b0afb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:04.341534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09847fdd-88dd-4b28-81b1-0698c3b0afb1 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:04.341819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09847fdd-88dd-4b28-81b1-0698c3b0afb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:04.342096 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09847fdd-88dd-4b28-81b1-0698c3b0afb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:04.401906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:04.409070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09847fdd-88dd-4b28-81b1-0698c3b0afb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:04.410581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09847fdd-88dd-4b28-81b1-0698c3b0afb1 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:04.505477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 626/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:36:04.511375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134f5d4-e520-4b1f-a659-cb9ca651c1ed None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:04.539454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:04.612446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134f5d4-e520-4b1f-a659-cb9ca651c1ed None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:04.646445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17da333-ac2d-4b05-85e7-a2d12a3eeead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:04.647064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17da333-ac2d-4b05-85e7-a2d12a3eeead None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:04.647344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17da333-ac2d-4b05-85e7-a2d12a3eeead None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:04.647616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17da333-ac2d-4b05-85e7-a2d12a3eeead None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:04.660192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17da333-ac2d-4b05-85e7-a2d12a3eeead None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:04.660192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17da333-ac2d-4b05-85e7-a2d12a3eeead None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-c17da333-ac2d-4b05-85e7-a2d12a3eeead None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:04.673128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:04.674270 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:04.675346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:04.675346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:04.675346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:04.675346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 616/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:04.680150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09847fdd-88dd-4b28-81b1-0698c3b0afb1 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:04.773260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:04.778930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09847fdd-88dd-4b28-81b1-0698c3b0afb1 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:04.813317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec096a-7055-4dee-a844-1b679cb3f6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:04.813927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec096a-7055-4dee-a844-1b679cb3f6fb None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:04.814212 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec096a-7055-4dee-a844-1b679cb3f6fb None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:04.814487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec096a-7055-4dee-a844-1b679cb3f6fb None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:04.815151 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec096a-7055-4dee-a844-1b679cb3f6fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:04.815455 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec096a-7055-4dee-a844-1b679cb3f6fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:04.816840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec096a-7055-4dee-a844-1b679cb3f6fb None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:04.827832 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 627/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:04.834185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17da333-ac2d-4b05-85e7-a2d12a3eeead None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:04.912525 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:04.932909 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17da333-ac2d-4b05-85e7-a2d12a3eeead None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:04.966079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e330af-19ea-4a26-bbc7-4e2092567ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:04.966697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e330af-19ea-4a26-bbc7-4e2092567ef8 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:04.966970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e330af-19ea-4a26-bbc7-4e2092567ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:04.967246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e330af-19ea-4a26-bbc7-4e2092567ef8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:04.974212 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e330af-19ea-4a26-bbc7-4e2092567ef8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:04.975807 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e330af-19ea-4a26-bbc7-4e2092567ef8 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-14e330af-19ea-4a26-bbc7-4e2092567ef8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:04.987028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:04.988080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:04.989028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:04.989028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:04.989028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:04.989028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:04.989028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:04.989028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:04.989028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:04.989028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:04.989028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 617/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:04.994218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec096a-7055-4dee-a844-1b679cb3f6fb None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:05.093788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec096a-7055-4dee-a844-1b679cb3f6fb None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:05.127762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22142ba-3711-4ea3-8851-eeb10ef6071c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:05.128354 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22142ba-3711-4ea3-8851-eeb10ef6071c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:05.128632 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22142ba-3711-4ea3-8851-eeb10ef6071c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:05.128947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22142ba-3711-4ea3-8851-eeb10ef6071c None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:05.129651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22142ba-3711-4ea3-8851-eeb10ef6071c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:05.129953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22142ba-3711-4ea3-8851-eeb10ef6071c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:05.131357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22142ba-3711-4ea3-8851-eeb10ef6071c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:05.143640 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 628/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:05.149742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e330af-19ea-4a26-bbc7-4e2092567ef8 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:05.247276 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e330af-19ea-4a26-bbc7-4e2092567ef8 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:05.279838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c95da-0498-4f72-a441-db6367c672e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:05.280429 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c95da-0498-4f72-a441-db6367c672e9 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:05.280738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c95da-0498-4f72-a441-db6367c672e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:05.281013 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c95da-0498-4f72-a441-db6367c672e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:05.281788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c95da-0498-4f72-a441-db6367c672e9 None admin] RBAC: Adding query filter params (role.id=3acdf08298c84325b53a0e286cc53b30, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=ea3816d5ae054741ac2c5a2eec3eb617) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:05.282098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c95da-0498-4f72-a441-db6367c672e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:05.283511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c95da-0498-4f72-a441-db6367c672e9 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:05.289094 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 618/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:05 2022] GET /identity/v3/role_assignments?user.id=ea3816d5ae054741ac2c5a2eec3eb617&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=3acdf08298c84325b53a0e286cc53b30 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:07.262007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d62c76-18a4-4ad8-a786-d4cc5b8628d8 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:07.262565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d62c76-18a4-4ad8-a786-d4cc5b8628d8 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:07.262565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d62c76-18a4-4ad8-a786-d4cc5b8628d8 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:07.263877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 629/1247] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:07.269325 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd19c9-6bc5-44b8-ab58-abb108c705d0 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:07.269767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd19c9-6bc5-44b8-ab58-abb108c705d0 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:07.269875 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd19c9-6bc5-44b8-ab58-abb108c705d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:07.276166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:07.288524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-09dd19c9-6bc5-44b8-ab58-abb108c705d0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:07.289785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-09dd19c9-6bc5-44b8-ab58-abb108c705d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:07.355862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 619/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:07.628044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d62c76-18a4-4ad8-a786-d4cc5b8628d8 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:07.647236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:07.728946 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d62c76-18a4-4ad8-a786-d4cc5b8628d8 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:07.763064 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118267a-a43f-4b85-9a2e-6903bfe70994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:07.763688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118267a-a43f-4b85-9a2e-6903bfe70994 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:07.763967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118267a-a43f-4b85-9a2e-6903bfe70994 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:07.764256 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118267a-a43f-4b85-9a2e-6903bfe70994 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:07.774741 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118267a-a43f-4b85-9a2e-6903bfe70994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-2118267a-a43f-4b85-9a2e-6903bfe70994 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:07.779484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:07.780582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:07.782864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:07.782864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:07.782864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:07.782864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:07.782864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:07.782864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:07.782864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:07.782864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:07.782864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:07.782864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:07.782864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:07.782864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:36:07.782864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 630/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:07.787214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd19c9-6bc5-44b8-ab58-abb108c705d0 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:07.883005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd19c9-6bc5-44b8-ab58-abb108c705d0 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:07.911708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:07.916631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183e8cd-4d9f-4e3d-bdbc-c3ec9a9eb5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:07.917252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183e8cd-4d9f-4e3d-bdbc-c3ec9a9eb5f0 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:07.917554 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183e8cd-4d9f-4e3d-bdbc-c3ec9a9eb5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:07.917856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183e8cd-4d9f-4e3d-bdbc-c3ec9a9eb5f0 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:07.918511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183e8cd-4d9f-4e3d-bdbc-c3ec9a9eb5f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:07.918814 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183e8cd-4d9f-4e3d-bdbc-c3ec9a9eb5f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:07.920170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183e8cd-4d9f-4e3d-bdbc-c3ec9a9eb5f0 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:07.928347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 620/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:07.934222 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118267a-a43f-4b85-9a2e-6903bfe70994 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:08.012410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:08.035009 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118267a-a43f-4b85-9a2e-6903bfe70994 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:08.068248 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5d2e8-6369-4696-98ff-6d09726de935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:08.068901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5d2e8-6369-4696-98ff-6d09726de935 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:08.069180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5d2e8-6369-4696-98ff-6d09726de935 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:08.069452 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5d2e8-6369-4696-98ff-6d09726de935 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:08.079923 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5d2e8-6369-4696-98ff-6d09726de935 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:08.081384 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5d2e8-6369-4696-98ff-6d09726de935 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-aff5d2e8-6369-4696-98ff-6d09726de935 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:08.095644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:08.101763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:08.102903 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:08.102903 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:08.102903 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:36:08.102903 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 631/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:08.103209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183e8cd-4d9f-4e3d-bdbc-c3ec9a9eb5f0 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:08.201625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183e8cd-4d9f-4e3d-bdbc-c3ec9a9eb5f0 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:08.235512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489887a-e8fb-46cf-864e-9d00ce131eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:08.236104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489887a-e8fb-46cf-864e-9d00ce131eb7 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:08.236385 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489887a-e8fb-46cf-864e-9d00ce131eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:08.236686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489887a-e8fb-46cf-864e-9d00ce131eb7 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:08.237318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489887a-e8fb-46cf-864e-9d00ce131eb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:08.237668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489887a-e8fb-46cf-864e-9d00ce131eb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:08.239023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489887a-e8fb-46cf-864e-9d00ce131eb7 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:08.250386 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 621/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:08.256092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5d2e8-6369-4696-98ff-6d09726de935 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:08.355497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5d2e8-6369-4696-98ff-6d09726de935 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:08.387682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:08.395769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7b0ef-0e10-4bc7-abd5-68af60d37a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:08.396394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7b0ef-0e10-4bc7-abd5-68af60d37a9e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:08.396701 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7b0ef-0e10-4bc7-abd5-68af60d37a9e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:08.396980 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7b0ef-0e10-4bc7-abd5-68af60d37a9e None admin] PATH_INFO: `/v3/users/placement` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:08.404827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7b0ef-0e10-4bc7-abd5-68af60d37a9e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:08.406267 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7b0ef-0e10-4bc7-abd5-68af60d37a9e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-01a7b0ef-0e10-4bc7-abd5-68af60d37a9e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:08.417999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:08.419130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:08.420335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:08.420335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:08.420335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:08.420335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:08.420335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:08.420335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:08.420335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:08.420335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:08.420335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:36:08.420335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:36:08.420335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 632/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:08.425028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489887a-e8fb-46cf-864e-9d00ce131eb7 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:08.480307 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:08.522804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489887a-e8fb-46cf-864e-9d00ce131eb7 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:08.556352 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887386ce-aba4-4387-8aea-4ca88d852ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:08.556983 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887386ce-aba4-4387-8aea-4ca88d852ea9 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:08.557263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887386ce-aba4-4387-8aea-4ca88d852ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:08.557533 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887386ce-aba4-4387-8aea-4ca88d852ea9 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:08.558252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887386ce-aba4-4387-8aea-4ca88d852ea9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:08.558558 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887386ce-aba4-4387-8aea-4ca88d852ea9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:08.559968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887386ce-aba4-4387-8aea-4ca88d852ea9 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:08.578149 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 622/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Aug 24 07:36:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:08.584695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7b0ef-0e10-4bc7-abd5-68af60d37a9e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:08.685154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7b0ef-0e10-4bc7-abd5-68af60d37a9e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:08.721345 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774e2e3-425f-47c5-8073-12c8e2bb43a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:08.721966 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774e2e3-425f-47c5-8073-12c8e2bb43a9 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:08.722256 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774e2e3-425f-47c5-8073-12c8e2bb43a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:08.722557 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774e2e3-425f-47c5-8073-12c8e2bb43a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:08.775999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:08.787690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774e2e3-425f-47c5-8073-12c8e2bb43a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:08.789176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774e2e3-425f-47c5-8073-12c8e2bb43a9 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:08.884991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 633/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:36:08.891547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887386ce-aba4-4387-8aea-4ca88d852ea9 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:08.988817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887386ce-aba4-4387-8aea-4ca88d852ea9 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:09.002101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:09.022241 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc62a7-01ba-4ea4-80f5-599d8eae5a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:09.022856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc62a7-01ba-4ea4-80f5-599d8eae5a10 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.023133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc62a7-01ba-4ea4-80f5-599d8eae5a10 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.023403 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc62a7-01ba-4ea4-80f5-599d8eae5a10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.032950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc62a7-01ba-4ea4-80f5-599d8eae5a10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:09.034374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc62a7-01ba-4ea4-80f5-599d8eae5a10 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-63bc62a7-01ba-4ea4-80f5-599d8eae5a10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:09.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:09.051788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:09.053550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:09.053550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:09.053550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:09.053550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 623/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:09.056691 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774e2e3-425f-47c5-8073-12c8e2bb43a9 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:09.142532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:09.153862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774e2e3-425f-47c5-8073-12c8e2bb43a9 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:09.187113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125c147-32a6-40e2-8654-6bac0919094d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:09.187771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c147-32a6-40e2-8654-6bac0919094d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.188704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c147-32a6-40e2-8654-6bac0919094d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.188981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c147-32a6-40e2-8654-6bac0919094d None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.189617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125c147-32a6-40e2-8654-6bac0919094d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:09.189920 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125c147-32a6-40e2-8654-6bac0919094d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:09.191278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125c147-32a6-40e2-8654-6bac0919094d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:09.202407 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 634/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:09.208393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc62a7-01ba-4ea4-80f5-599d8eae5a10 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:09.307251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc62a7-01ba-4ea4-80f5-599d8eae5a10 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:09.340390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b96ba-8eda-45c4-bc60-3cfde0324387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:09.341050 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b96ba-8eda-45c4-bc60-3cfde0324387 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.341329 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b96ba-8eda-45c4-bc60-3cfde0324387 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.341601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b96ba-8eda-45c4-bc60-3cfde0324387 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.348858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b96ba-8eda-45c4-bc60-3cfde0324387 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:09.353275 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b96ba-8eda-45c4-bc60-3cfde0324387 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-a14b96ba-8eda-45c4-bc60-3cfde0324387 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:09.361663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:09.362727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:09.363834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:09.363834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:09.363834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:09.363834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:09.363834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:09.363834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:09.363834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:09.363834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:09.363834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 624/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:09.368874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125c147-32a6-40e2-8654-6bac0919094d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:09.464578 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125c147-32a6-40e2-8654-6bac0919094d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:09.497788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4e966-bd66-438b-8d66-dba393e33bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:09.498383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4e966-bd66-438b-8d66-dba393e33bba None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.498662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4e966-bd66-438b-8d66-dba393e33bba None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.498932 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4e966-bd66-438b-8d66-dba393e33bba None admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.499606 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4e966-bd66-438b-8d66-dba393e33bba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:09.499910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4e966-bd66-438b-8d66-dba393e33bba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:09.501554 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4e966-bd66-438b-8d66-dba393e33bba None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:09.513378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 635/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:09.519713 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b96ba-8eda-45c4-bc60-3cfde0324387 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:09.545648 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:09.620046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b96ba-8eda-45c4-bc60-3cfde0324387 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:09.654187 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a9b28-e28e-4783-bb15-1a6272df4153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:09.654826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a9b28-e28e-4783-bb15-1a6272df4153 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.655105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a9b28-e28e-4783-bb15-1a6272df4153 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.655368 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a9b28-e28e-4783-bb15-1a6272df4153 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.656083 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a9b28-e28e-4783-bb15-1a6272df4153 None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=ea3816d5ae054741ac2c5a2eec3eb617) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:09.656390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a9b28-e28e-4783-bb15-1a6272df4153 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:09.658003 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a9b28-e28e-4783-bb15-1a6272df4153 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:09.663319 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 625/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:09 2022] GET /identity/v3/role_assignments?user.id=ea3816d5ae054741ac2c5a2eec3eb617&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:11.559597 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b66d4-bd55-4f33-9388-5836c8d3efb9 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:11.560095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b66d4-bd55-4f33-9388-5836c8d3efb9 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:11.560188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b66d4-bd55-4f33-9388-5836c8d3efb9 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:11.561513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 636/1261] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:11.567422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fed74-1e2f-4302-a23b-7afdf80610eb None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:11.567805 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fed74-1e2f-4302-a23b-7afdf80610eb None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:11.567967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fed74-1e2f-4302-a23b-7afdf80610eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:11.586317 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-ae5fed74-1e2f-4302-a23b-7afdf80610eb None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:11.587493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ae5fed74-1e2f-4302-a23b-7afdf80610eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:11.654612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 626/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:11.927325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b66d4-bd55-4f33-9388-5836c8d3efb9 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:11.938998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:12.028153 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b66d4-bd55-4f33-9388-5836c8d3efb9 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:12.062501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6495e70-cc42-4f2d-a951-0d547992cfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:12.063770 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6495e70-cc42-4f2d-a951-0d547992cfbc None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:12.064050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6495e70-cc42-4f2d-a951-0d547992cfbc None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:12.064326 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6495e70-cc42-4f2d-a951-0d547992cfbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:12.074263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6495e70-cc42-4f2d-a951-0d547992cfbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-f6495e70-cc42-4f2d-a951-0d547992cfbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:12.078918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:12.079986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:12.081655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:12.081655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:12.081655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:12.081655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:12.081655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:12.081655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:12.081655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:12.081655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:12.081655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:12.081655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:12.081655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:12.081655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:36:12.081655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 637/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:12.086061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fed74-1e2f-4302-a23b-7afdf80610eb None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:12.104598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:12.183030 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fed74-1e2f-4302-a23b-7afdf80610eb None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:12.215962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce86411-6e23-4e77-8673-e82e0da22047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:12.216553 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce86411-6e23-4e77-8673-e82e0da22047 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:12.216860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce86411-6e23-4e77-8673-e82e0da22047 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:12.217136 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce86411-6e23-4e77-8673-e82e0da22047 None admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:12.217790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce86411-6e23-4e77-8673-e82e0da22047 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:12.218094 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce86411-6e23-4e77-8673-e82e0da22047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:12.219551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce86411-6e23-4e77-8673-e82e0da22047 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:12.228118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 627/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:12.233657 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6495e70-cc42-4f2d-a951-0d547992cfbc None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:12.302059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:12.338305 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6495e70-cc42-4f2d-a951-0d547992cfbc None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:12.372907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cd8fa-4a2d-43f7-9104-360053fc835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:12.373534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cd8fa-4a2d-43f7-9104-360053fc835c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:12.373819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cd8fa-4a2d-43f7-9104-360053fc835c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:12.374094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cd8fa-4a2d-43f7-9104-360053fc835c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:12.384779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cd8fa-4a2d-43f7-9104-360053fc835c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:12.386283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cd8fa-4a2d-43f7-9104-360053fc835c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-f45cd8fa-4a2d-43f7-9104-360053fc835c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:12.400018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:12.401839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:12.402802 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:12.402802 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:12.402802 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:36:12.402802 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 638/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:12.407701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce86411-6e23-4e77-8673-e82e0da22047 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:12.489692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:12.508336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce86411-6e23-4e77-8673-e82e0da22047 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:12.541685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad5808-94a6-4dda-88c8-36e6f92b1c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:12.542304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad5808-94a6-4dda-88c8-36e6f92b1c35 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:12.542586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad5808-94a6-4dda-88c8-36e6f92b1c35 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:12.542862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad5808-94a6-4dda-88c8-36e6f92b1c35 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:12.543500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad5808-94a6-4dda-88c8-36e6f92b1c35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:12.543806 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad5808-94a6-4dda-88c8-36e6f92b1c35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:12.545525 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad5808-94a6-4dda-88c8-36e6f92b1c35 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:12.557998 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 628/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:12.563874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cd8fa-4a2d-43f7-9104-360053fc835c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:12.663166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cd8fa-4a2d-43f7-9104-360053fc835c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:12.684233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:12.698209 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38d0b1-ee28-43bd-9f30-0e5b384216f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:12.698840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38d0b1-ee28-43bd-9f30-0e5b384216f5 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:12.699123 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38d0b1-ee28-43bd-9f30-0e5b384216f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:12.699397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38d0b1-ee28-43bd-9f30-0e5b384216f5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:12.707407 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38d0b1-ee28-43bd-9f30-0e5b384216f5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:12.712869 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38d0b1-ee28-43bd-9f30-0e5b384216f5 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-fd38d0b1-ee28-43bd-9f30-0e5b384216f5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:12.720560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:12.721982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:12.723013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:12.723013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:12.723013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:12.723013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:12.723013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:12.723013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:12.723013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:12.723013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:12.723013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:36:12.723013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:36:12.723013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 639/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:12.727856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad5808-94a6-4dda-88c8-36e6f92b1c35 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:12.827278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad5808-94a6-4dda-88c8-36e6f92b1c35 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:12.861715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2283f30-6589-4ee1-ab3b-d905934cab40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:12.862307 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2283f30-6589-4ee1-ab3b-d905934cab40 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:12.862584 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2283f30-6589-4ee1-ab3b-d905934cab40 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:12.862854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2283f30-6589-4ee1-ab3b-d905934cab40 None admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:12.863563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2283f30-6589-4ee1-ab3b-d905934cab40 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:12.863867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2283f30-6589-4ee1-ab3b-d905934cab40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:12.865319 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2283f30-6589-4ee1-ab3b-d905934cab40 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:12.874023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:12.881917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 629/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Aug 24 07:36:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:12.887753 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38d0b1-ee28-43bd-9f30-0e5b384216f5 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:12.988323 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38d0b1-ee28-43bd-9f30-0e5b384216f5 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:13.022008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8e4e-56ab-4e4d-a134-8316abe504f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:13.022623 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8e4e-56ab-4e4d-a134-8316abe504f9 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:13.022898 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8e4e-56ab-4e4d-a134-8316abe504f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:13.023178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8e4e-56ab-4e4d-a134-8316abe504f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:13.034135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea8e4e-56ab-4e4d-a134-8316abe504f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:13.035562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea8e4e-56ab-4e4d-a134-8316abe504f9 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-8aea8e4e-56ab-4e4d-a134-8316abe504f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:13.049981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:13.051098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:13.052198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:13.052198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:13.052198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:36:13.052198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 640/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:13.057086 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2283f30-6589-4ee1-ab3b-d905934cab40 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:13.155334 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2283f30-6589-4ee1-ab3b-d905934cab40 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:13.189746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85011b-42dc-4b28-9f10-8fc35f716f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:13.190340 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85011b-42dc-4b28-9f10-8fc35f716f2b None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:13.190617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85011b-42dc-4b28-9f10-8fc35f716f2b None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:13.190890 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85011b-42dc-4b28-9f10-8fc35f716f2b None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:13.191542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85011b-42dc-4b28-9f10-8fc35f716f2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:13.191841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85011b-42dc-4b28-9f10-8fc35f716f2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:13.193326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85011b-42dc-4b28-9f10-8fc35f716f2b None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:13.204427 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 630/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:13.210263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8e4e-56ab-4e4d-a134-8316abe504f9 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:13.242361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:13.309246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8e4e-56ab-4e4d-a134-8316abe504f9 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:13.342724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdaa41-f27e-45f1-af0f-226053dfbf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:13.343346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdaa41-f27e-45f1-af0f-226053dfbf7f None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:13.343626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdaa41-f27e-45f1-af0f-226053dfbf7f None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:13.344549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdaa41-f27e-45f1-af0f-226053dfbf7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:13.351343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdaa41-f27e-45f1-af0f-226053dfbf7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:13.352845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdaa41-f27e-45f1-af0f-226053dfbf7f None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-ebfdaa41-f27e-45f1-af0f-226053dfbf7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:13.364511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:13.365633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:13.366882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:13.366882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:13.366882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:13.366882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:13.366882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:13.366882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:13.366882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:13.366882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:36:13.366882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 641/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:13.371551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85011b-42dc-4b28-9f10-8fc35f716f2b None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:13.429940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:13.471774 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85011b-42dc-4b28-9f10-8fc35f716f2b None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:13.507163 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de274279-7f51-4583-9ad7-fcbee0bac2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:13.507767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de274279-7f51-4583-9ad7-fcbee0bac2ee None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:13.508047 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de274279-7f51-4583-9ad7-fcbee0bac2ee None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:13.508322 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de274279-7f51-4583-9ad7-fcbee0bac2ee None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:13.509151 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de274279-7f51-4583-9ad7-fcbee0bac2ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:13.509460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de274279-7f51-4583-9ad7-fcbee0bac2ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:13.510875 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de274279-7f51-4583-9ad7-fcbee0bac2ee None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:13.523015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 631/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:13.529121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdaa41-f27e-45f1-af0f-226053dfbf7f None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:13.619968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:13.628369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdaa41-f27e-45f1-af0f-226053dfbf7f None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:13.661618 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af833587-a3fd-419d-b899-b7d7cf52938a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:13.662279 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af833587-a3fd-419d-b899-b7d7cf52938a None admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:13.662561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af833587-a3fd-419d-b899-b7d7cf52938a None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:13.662839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af833587-a3fd-419d-b899-b7d7cf52938a None admin] PATH_INFO: `/v3/projects/fc060764023e44a7ade80eeed98a235f/users/ea3816d5ae054741ac2c5a2eec3eb617/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:13.664790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af833587-a3fd-419d-b899-b7d7cf52938a None admin] RBAC: Authorizing `identity:create_grant(project_id=fc060764023e44a7ade80eeed98a235f, user_id=ea3816d5ae054741ac2c5a2eec3eb617, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:13.666246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af833587-a3fd-419d-b899-b7d7cf52938a None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:13.810751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 642/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 24 07:36:13 2022] PUT /identity/v3/projects/fc060764023e44a7ade80eeed98a235f/users/ea3816d5ae054741ac2c5a2eec3eb617/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:15.705017 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4d5eb-d024-4709-b4f0-2f5333511280 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:15.705545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4d5eb-d024-4709-b4f0-2f5333511280 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:15.705545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4d5eb-d024-4709-b4f0-2f5333511280 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:15.706892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 632/1274] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:36:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:15.712026 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90fab4-64fb-4ea0-b9e1-d3c781855bf4 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:15.712592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90fab4-64fb-4ea0-b9e1-d3c781855bf4 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:15.712592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90fab4-64fb-4ea0-b9e1-d3c781855bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:15.731119 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-6f90fab4-64fb-4ea0-b9e1-d3c781855bf4 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:15.732332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6f90fab4-64fb-4ea0-b9e1-d3c781855bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:15.799685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 643/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:36:16.067278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4d5eb-d024-4709-b4f0-2f5333511280 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:16.168905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4d5eb-d024-4709-b4f0-2f5333511280 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:16.185747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:16.202824 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92287da9-6f35-4399-b0c8-be128ece2758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:16.204059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92287da9-6f35-4399-b0c8-be128ece2758 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:16.204336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92287da9-6f35-4399-b0c8-be128ece2758 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:16.204611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92287da9-6f35-4399-b0c8-be128ece2758 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:16.214366 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92287da9-6f35-4399-b0c8-be128ece2758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-92287da9-6f35-4399-b0c8-be128ece2758 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:16.219011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:16.220135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:16.221458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:16.221458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:16.221458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:16.221458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:16.221458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:16.221458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:16.221458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:16.221458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:16.221458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:16.221458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:16.221458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:16.221458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:16.221458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 633/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:16.226638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90fab4-64fb-4ea0-b9e1-d3c781855bf4 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:16.305111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:16.322983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90fab4-64fb-4ea0-b9e1-d3c781855bf4 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:16.355562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb5d27-1709-4151-91da-85b832a08e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:16.356156 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb5d27-1709-4151-91da-85b832a08e6e None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:16.356433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb5d27-1709-4151-91da-85b832a08e6e None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:16.356731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb5d27-1709-4151-91da-85b832a08e6e None admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:16.357381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb5d27-1709-4151-91da-85b832a08e6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:16.357686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb5d27-1709-4151-91da-85b832a08e6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:16.359034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb5d27-1709-4151-91da-85b832a08e6e None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:16.366979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 644/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:16.372872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92287da9-6f35-4399-b0c8-be128ece2758 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:16.474834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92287da9-6f35-4399-b0c8-be128ece2758 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:16.507809 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0e9ea-5335-4773-8bf4-82895d0fbcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:16.508423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0e9ea-5335-4773-8bf4-82895d0fbcba None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:16.508726 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0e9ea-5335-4773-8bf4-82895d0fbcba None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:16.508997 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0e9ea-5335-4773-8bf4-82895d0fbcba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:16.519384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0e9ea-5335-4773-8bf4-82895d0fbcba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:16.520838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0e9ea-5335-4773-8bf4-82895d0fbcba None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-19a0e9ea-5335-4773-8bf4-82895d0fbcba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:16.535835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:16.537010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:16.538118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:16.538118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:16.538118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:16.538118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 634/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:16.543048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb5d27-1709-4151-91da-85b832a08e6e None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:16.642006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb5d27-1709-4151-91da-85b832a08e6e None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:16.674548 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d5ed3-6590-4f95-9b78-84cabfdee61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:16.675147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d5ed3-6590-4f95-9b78-84cabfdee61a None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:16.675435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d5ed3-6590-4f95-9b78-84cabfdee61a None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:16.675699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d5ed3-6590-4f95-9b78-84cabfdee61a None admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:16.676327 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d5ed3-6590-4f95-9b78-84cabfdee61a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:16.676631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d5ed3-6590-4f95-9b78-84cabfdee61a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:16.678072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d5ed3-6590-4f95-9b78-84cabfdee61a None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:16.689535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 645/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:16.695446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0e9ea-5335-4773-8bf4-82895d0fbcba None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:16.726690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:16.795395 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0e9ea-5335-4773-8bf4-82895d0fbcba None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:16.828918 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eeab1d-1e23-4fbe-b6f1-c7c0f9f09896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:16.829582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eeab1d-1e23-4fbe-b6f1-c7c0f9f09896 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:16.829860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eeab1d-1e23-4fbe-b6f1-c7c0f9f09896 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:16.830129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eeab1d-1e23-4fbe-b6f1-c7c0f9f09896 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:16.836895 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eeab1d-1e23-4fbe-b6f1-c7c0f9f09896 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:16.838331 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eeab1d-1e23-4fbe-b6f1-c7c0f9f09896 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-f0eeab1d-1e23-4fbe-b6f1-c7c0f9f09896 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:16.849616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:16.850757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:16.851906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:16.851906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:16.851906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:16.851906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:16.851906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:16.851906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:16.851906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:16.851906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:16.851906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:36:16.851906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:16.851906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 635/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:16.856782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d5ed3-6590-4f95-9b78-84cabfdee61a None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:16.860921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:16.954982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d5ed3-6590-4f95-9b78-84cabfdee61a None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:16.987147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94077b-5be5-4ffc-8fdf-88837590a01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:16.987745 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94077b-5be5-4ffc-8fdf-88837590a01c None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:16.988022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94077b-5be5-4ffc-8fdf-88837590a01c None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:16.988289 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94077b-5be5-4ffc-8fdf-88837590a01c None admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:16.989031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94077b-5be5-4ffc-8fdf-88837590a01c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:16.989332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94077b-5be5-4ffc-8fdf-88837590a01c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:16.990738 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94077b-5be5-4ffc-8fdf-88837590a01c None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:17.009336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 646/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Aug 24 07:36:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:17.015269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eeab1d-1e23-4fbe-b6f1-c7c0f9f09896 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:17.104664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:17.114584 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eeab1d-1e23-4fbe-b6f1-c7c0f9f09896 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:17.150199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adb7f7-3a9b-451b-b062-c2a2df52ad1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:17.150793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adb7f7-3a9b-451b-b062-c2a2df52ad1e None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:17.151076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adb7f7-3a9b-451b-b062-c2a2df52ad1e None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:17.151353 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adb7f7-3a9b-451b-b062-c2a2df52ad1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:17.220070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86adb7f7-3a9b-451b-b062-c2a2df52ad1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:17.221744 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86adb7f7-3a9b-451b-b062-c2a2df52ad1e None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:17.311651 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:17.321693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 636/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:36:17.327455 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94077b-5be5-4ffc-8fdf-88837590a01c None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:17.387197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:17.424716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94077b-5be5-4ffc-8fdf-88837590a01c None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:17.457253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8e48c-1811-4f47-9057-3b869638993d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:17.457875 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8e48c-1811-4f47-9057-3b869638993d None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:17.458156 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8e48c-1811-4f47-9057-3b869638993d None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:17.458428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8e48c-1811-4f47-9057-3b869638993d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:17.467985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8e48c-1811-4f47-9057-3b869638993d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:17.470032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8e48c-1811-4f47-9057-3b869638993d None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-60a8e48c-1811-4f47-9057-3b869638993d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:17.484325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:17.486046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:17.486974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:17.486974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:17.486974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:36:17.486974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 647/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:17.491695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adb7f7-3a9b-451b-b062-c2a2df52ad1e None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:17.589433 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adb7f7-3a9b-451b-b062-c2a2df52ad1e None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:17.624682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a326-0266-4a17-9cf3-7673d96c5400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:17.625854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a326-0266-4a17-9cf3-7673d96c5400 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:17.626132 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a326-0266-4a17-9cf3-7673d96c5400 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:17.626411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a326-0266-4a17-9cf3-7673d96c5400 None admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:17.627040 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a326-0266-4a17-9cf3-7673d96c5400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:17.627343 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a326-0266-4a17-9cf3-7673d96c5400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:17.628732 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a326-0266-4a17-9cf3-7673d96c5400 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:17.639668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 637/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:17.646099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8e48c-1811-4f47-9057-3b869638993d None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:17.753182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8e48c-1811-4f47-9057-3b869638993d None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:17.767870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:17.789071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2fb17-ad7b-48a9-8501-5adf580ff2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:17.789719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2fb17-ad7b-48a9-8501-5adf580ff2c1 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:17.790001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2fb17-ad7b-48a9-8501-5adf580ff2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:17.790270 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2fb17-ad7b-48a9-8501-5adf580ff2c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:17.798232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2fb17-ad7b-48a9-8501-5adf580ff2c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:17.799806 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2fb17-ad7b-48a9-8501-5adf580ff2c1 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application [None req-16f2fb17-ad7b-48a9-8501-5adf580ff2c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:17.812712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:17.819145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:17.820375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:17.820375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:17.820375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:17.820375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:17.820375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:17.820375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:17.820375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:17.820375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: ERROR keystone.server.flask.application Aug 24 07:36:17.820375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 648/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:17.820946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a326-0266-4a17-9cf3-7673d96c5400 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:17.873552 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:17.929953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a326-0266-4a17-9cf3-7673d96c5400 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:17.966863 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72bb97-7bd3-4abc-ada1-1cf35c6c94f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:17.967581 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72bb97-7bd3-4abc-ada1-1cf35c6c94f0 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:17.967932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72bb97-7bd3-4abc-ada1-1cf35c6c94f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:17.968264 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72bb97-7bd3-4abc-ada1-1cf35c6c94f0 None admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:17.969743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72bb97-7bd3-4abc-ada1-1cf35c6c94f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:17.970068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72bb97-7bd3-4abc-ada1-1cf35c6c94f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:17.971784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72bb97-7bd3-4abc-ada1-1cf35c6c94f0 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:17.985251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 638/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:17.991601 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2fb17-ad7b-48a9-8501-5adf580ff2c1 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:18.099006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2fb17-ad7b-48a9-8501-5adf580ff2c1 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:18.133420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf55da-1406-47a4-bd7a-f7bcf830f714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:18.134062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf55da-1406-47a4-bd7a-f7bcf830f714 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:18.134336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf55da-1406-47a4-bd7a-f7bcf830f714 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:18.134727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf55da-1406-47a4-bd7a-f7bcf830f714 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:18.135509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf55da-1406-47a4-bd7a-f7bcf830f714 None admin] RBAC: Adding query filter params (role.id=8ea69cbce665406aa23d4e7e90096e20, scope.project.id=fc060764023e44a7ade80eeed98a235f, user.id=ea3816d5ae054741ac2c5a2eec3eb617) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:18.135823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf55da-1406-47a4-bd7a-f7bcf830f714 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:18.137655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf55da-1406-47a4-bd7a-f7bcf830f714 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:18.144574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 649/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:17 2022] GET /identity/v3/role_assignments?user.id=ea3816d5ae054741ac2c5a2eec3eb617&scope.project.id=fc060764023e44a7ade80eeed98a235f&role.id=8ea69cbce665406aa23d4e7e90096e20 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:20.048946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958afab3-50bd-4ca6-8cb5-cff654f12394 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.049519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958afab3-50bd-4ca6-8cb5-cff654f12394 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.049519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958afab3-50bd-4ca6-8cb5-cff654f12394 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.050815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 639/1288] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:36:20.056107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63567941-fa19-465d-bb41-6ab6d15b79f8 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.056548 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63567941-fa19-465d-bb41-6ab6d15b79f8 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.056675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63567941-fa19-465d-bb41-6ab6d15b79f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.074950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-63567941-fa19-465d-bb41-6ab6d15b79f8 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:20.078995 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-63567941-fa19-465d-bb41-6ab6d15b79f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:20.105310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:20.140402 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 650/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:20.405820 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958afab3-50bd-4ca6-8cb5-cff654f12394 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:20.506079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958afab3-50bd-4ca6-8cb5-cff654f12394 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:20.516492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:20.539136 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e55565-5561-4b51-83ba-9b044fec94fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:20.539759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e55565-5561-4b51-83ba-9b044fec94fe None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.540032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e55565-5561-4b51-83ba-9b044fec94fe None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.540298 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e55565-5561-4b51-83ba-9b044fec94fe None admin] PATH_INFO: `/v3/services/placement` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.540947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e55565-5561-4b51-83ba-9b044fec94fe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:20.542298 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e55565-5561-4b51-83ba-9b044fec94fe None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-25e55565-5561-4b51-83ba-9b044fec94fe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:20.551089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:20.552281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:20.553198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 640/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 24 07:36:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:20.558197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63567941-fa19-465d-bb41-6ab6d15b79f8 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:20.655727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63567941-fa19-465d-bb41-6ab6d15b79f8 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:20.690606 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda11561-5edc-4d17-8bf6-838d2ac26be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:20.691142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda11561-5edc-4d17-8bf6-838d2ac26be0 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.691422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda11561-5edc-4d17-8bf6-838d2ac26be0 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.691694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda11561-5edc-4d17-8bf6-838d2ac26be0 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.692317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda11561-5edc-4d17-8bf6-838d2ac26be0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:20.692622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda11561-5edc-4d17-8bf6-838d2ac26be0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:20.694016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda11561-5edc-4d17-8bf6-838d2ac26be0 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:20.698783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 651/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Aug 24 07:36:20 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:20.705417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e55565-5561-4b51-83ba-9b044fec94fe None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:20.802190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e55565-5561-4b51-83ba-9b044fec94fe None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:20.834170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b60435-f28d-4624-9c30-8bc35220e3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:20.834762 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b60435-f28d-4624-9c30-8bc35220e3aa None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.835123 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b60435-f28d-4624-9c30-8bc35220e3aa None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.835396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b60435-f28d-4624-9c30-8bc35220e3aa None admin] PATH_INFO: `/v3/services` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.836015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b60435-f28d-4624-9c30-8bc35220e3aa None admin] RBAC: Adding query filter params (type=placement) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:20.836315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b60435-f28d-4624-9c30-8bc35220e3aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:20.837686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b60435-f28d-4624-9c30-8bc35220e3aa None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:20.841936 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 641/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Aug 24 07:36:20 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:22.752463 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00cc8a-bc19-40dc-9f37-39fa0817b1ac None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:22.753020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00cc8a-bc19-40dc-9f37-39fa0817b1ac None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:22.753020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00cc8a-bc19-40dc-9f37-39fa0817b1ac None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:22.754356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 652/1293] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:22.759626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef713fca-1787-4928-8aad-a5cf7e9b2ea4 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:22.760070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef713fca-1787-4928-8aad-a5cf7e9b2ea4 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:22.760157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef713fca-1787-4928-8aad-a5cf7e9b2ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:22.778331 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-ef713fca-1787-4928-8aad-a5cf7e9b2ea4 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:22.779487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ef713fca-1787-4928-8aad-a5cf7e9b2ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:22.808994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:22.845483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 642/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:23.116273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00cc8a-bc19-40dc-9f37-39fa0817b1ac None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:23.147879 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:23.215313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00cc8a-bc19-40dc-9f37-39fa0817b1ac None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:23.248748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c3ebb-2776-482e-b567-e2d7451d7cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:23.249340 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c3ebb-2776-482e-b567-e2d7451d7cc3 None admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:23.249616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c3ebb-2776-482e-b567-e2d7451d7cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:23.249921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c3ebb-2776-482e-b567-e2d7451d7cc3 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:23.250704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c3ebb-2776-482e-b567-e2d7451d7cc3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:23.252093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c3ebb-2776-482e-b567-e2d7451d7cc3 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:23.335853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 653/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 24 07:36:23 2022] POST /identity/v3/services => generated 246 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:36:25.243919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02657ab1-fe5b-47db-a249-a176830b3525 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:25.244473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02657ab1-fe5b-47db-a249-a176830b3525 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:25.244473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02657ab1-fe5b-47db-a249-a176830b3525 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:25.245840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 643/1296] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:25.251664 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4899499-0c5f-45f4-b00d-0f6febde44f2 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:25.252058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4899499-0c5f-45f4-b00d-0f6febde44f2 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:25.252204 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4899499-0c5f-45f4-b00d-0f6febde44f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:25.271074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-d4899499-0c5f-45f4-b00d-0f6febde44f2 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:25.272253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d4899499-0c5f-45f4-b00d-0f6febde44f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:25.343934 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 654/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:25 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:25.611752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02657ab1-fe5b-47db-a249-a176830b3525 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:25.710214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02657ab1-fe5b-47db-a249-a176830b3525 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:25.743689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46af6d-d377-4e6d-9efc-e52cdd12ac05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:25.744300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46af6d-d377-4e6d-9efc-e52cdd12ac05 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:25.744580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46af6d-d377-4e6d-9efc-e52cdd12ac05 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:25.744881 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46af6d-d377-4e6d-9efc-e52cdd12ac05 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:25.745459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46af6d-d377-4e6d-9efc-e52cdd12ac05 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:25.746852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46af6d-d377-4e6d-9efc-e52cdd12ac05 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-3e46af6d-d377-4e6d-9efc-e52cdd12ac05 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:25.755516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:25.756789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:25.759899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 644/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 24 07:36:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:25.762596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4899499-0c5f-45f4-b00d-0f6febde44f2 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:25.780936 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:25.858934 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4899499-0c5f-45f4-b00d-0f6febde44f2 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:25.892203 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a9e3c-e6a3-4f84-bff7-60d9fdd794a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:25.892892 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9e3c-e6a3-4f84-bff7-60d9fdd794a5 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:25.893172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9e3c-e6a3-4f84-bff7-60d9fdd794a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:25.893441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9e3c-e6a3-4f84-bff7-60d9fdd794a5 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:25.894059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a9e3c-e6a3-4f84-bff7-60d9fdd794a5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:25.894357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a9e3c-e6a3-4f84-bff7-60d9fdd794a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:25.895704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a9e3c-e6a3-4f84-bff7-60d9fdd794a5 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:25.901700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 655/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Aug 24 07:36:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:25.906658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46af6d-d377-4e6d-9efc-e52cdd12ac05 None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:25.937890 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:26.005345 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46af6d-d377-4e6d-9efc-e52cdd12ac05 None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:26.040008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc941887-86c7-4fc8-944b-8101608b9fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:26.040682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc941887-86c7-4fc8-944b-8101608b9fc2 None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:26.040968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc941887-86c7-4fc8-944b-8101608b9fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:26.041237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc941887-86c7-4fc8-944b-8101608b9fc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:26.041919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc941887-86c7-4fc8-944b-8101608b9fc2 None admin] RBAC: Adding query filter params (interface=public, service_id=265af30e0ffc43a2a43f716b4e1cd913, region_id=RegionOne) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:26.042225 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc941887-86c7-4fc8-944b-8101608b9fc2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:26.043614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc941887-86c7-4fc8-944b-8101608b9fc2 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:26.048720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 645/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 24 07:36:25 2022] GET /identity/v3/endpoints?service_id=265af30e0ffc43a2a43f716b4e1cd913&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:26.054494 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a9e3c-e6a3-4f84-bff7-60d9fdd794a5 None admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:26.134333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:26.151621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a9e3c-e6a3-4f84-bff7-60d9fdd794a5 None admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:26.184213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1d543-e9ed-48b3-80ce-98c29e4010a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:26.184835 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1d543-e9ed-48b3-80ce-98c29e4010a5 None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:26.185114 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1d543-e9ed-48b3-80ce-98c29e4010a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:26.185384 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1d543-e9ed-48b3-80ce-98c29e4010a5 None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:26.185974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1d543-e9ed-48b3-80ce-98c29e4010a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:26.187424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1d543-e9ed-48b3-80ce-98c29e4010a5 None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:26.194645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 656/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 24 07:36:26 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:36:28.109810 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725faea0-124a-4f0e-b431-1f4a96e0d019 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.110352 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725faea0-124a-4f0e-b431-1f4a96e0d019 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.110352 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725faea0-124a-4f0e-b431-1f4a96e0d019 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.111693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 646/1302] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:28.117219 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe0cf4-2866-4850-a20f-0a49ad1f1cba None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.117651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe0cf4-2866-4850-a20f-0a49ad1f1cba None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.117767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe0cf4-2866-4850-a20f-0a49ad1f1cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.136081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-2bbe0cf4-2866-4850-a20f-0a49ad1f1cba None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:28.137293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2bbe0cf4-2866-4850-a20f-0a49ad1f1cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:28.202216 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 657/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:28 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:28.482484 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725faea0-124a-4f0e-b431-1f4a96e0d019 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.577359 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:28.584030 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725faea0-124a-4f0e-b431-1f4a96e0d019 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.616959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ebf64-a2e4-4a1b-91fd-354d830cca7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.617578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ebf64-a2e4-4a1b-91fd-354d830cca7c None admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.617853 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ebf64-a2e4-4a1b-91fd-354d830cca7c None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.620591 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ebf64-a2e4-4a1b-91fd-354d830cca7c None admin] PATH_INFO: `/v3/services/placement` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.620591 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ebf64-a2e4-4a1b-91fd-354d830cca7c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.620591 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ebf64-a2e4-4a1b-91fd-354d830cca7c None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-386ebf64-a2e4-4a1b-91fd-354d830cca7c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:28.629197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:28.630374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 07:36:28.631448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 647/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 24 07:36:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:28.636476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe0cf4-2866-4850-a20f-0a49ad1f1cba None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.711511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:28.734080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe0cf4-2866-4850-a20f-0a49ad1f1cba None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.767328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd51b24-d2bb-4f1a-9657-feac9059a4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.767924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd51b24-d2bb-4f1a-9657-feac9059a4fd None admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.768204 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd51b24-d2bb-4f1a-9657-feac9059a4fd None admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.768477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd51b24-d2bb-4f1a-9657-feac9059a4fd None admin] PATH_INFO: `/v3/services` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.769146 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd51b24-d2bb-4f1a-9657-feac9059a4fd None admin] RBAC: Adding query filter params (name=placement) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:28.769661 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd51b24-d2bb-4f1a-9657-feac9059a4fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.771346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd51b24-d2bb-4f1a-9657-feac9059a4fd None admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:28.776693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 658/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Aug 24 07:36:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:28.783531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ebf64-a2e4-4a1b-91fd-354d830cca7c None admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.886281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ebf64-a2e4-4a1b-91fd-354d830cca7c None admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.920072 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96652d9f-9614-4ba7-8fe3-055efafcbb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.920679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96652d9f-9614-4ba7-8fe3-055efafcbb18 None admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.920964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96652d9f-9614-4ba7-8fe3-055efafcbb18 None admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.921232 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96652d9f-9614-4ba7-8fe3-055efafcbb18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.922027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96652d9f-9614-4ba7-8fe3-055efafcbb18 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.923639 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96652d9f-9614-4ba7-8fe3-055efafcbb18 None admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:29.027962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 648/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:36:28 2022] POST /identity/v3/endpoints => generated 331 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:36:38.773742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=30 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:39.023248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:33.128005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d3709-d974-4c7b-9b75-de3d04f1d585 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:33.128599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d3709-d974-4c7b-9b75-de3d04f1d585 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:33.128599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d3709-d974-4c7b-9b75-de3d04f1d585 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:33.135299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158a472-a900-49ff-afd5-97eff4048b10 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:33.135740 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158a472-a900-49ff-afd5-97eff4048b10 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:33.135836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158a472-a900-49ff-afd5-97eff4048b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:33.138766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 659/1307] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:43:33 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:43:33.225844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-2158a472-a900-49ff-afd5-97eff4048b10 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:33.227070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2158a472-a900-49ff-afd5-97eff4048b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:33.365163 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 649/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:43:33.651763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca4ebb-638f-4a33-ba29-b95acae8086b None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:33.652291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca4ebb-638f-4a33-ba29-b95acae8086b None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:33.652734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca4ebb-638f-4a33-ba29-b95acae8086b None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:33.653993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 660/1309] 10.222.0.26 () {54 vars in 965 bytes} [Wed Aug 24 07:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:43:33.663933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233907a3-8181-4951-8e3f-4ca1c8fac7d4 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:33.664371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233907a3-8181-4951-8e3f-4ca1c8fac7d4 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:33.666167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233907a3-8181-4951-8e3f-4ca1c8fac7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:33.758885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-233907a3-8181-4951-8e3f-4ca1c8fac7d4 None None] MFA Rules not processed for user `46e61402c01c4f9ba5f18e94960bc313`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:33.760271 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-233907a3-8181-4951-8e3f-4ca1c8fac7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:33.784474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:33.854001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 650/1310] 10.222.0.26 () {58 vars in 1062 bytes} [Wed Aug 24 07:43:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:33.864419 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca4ebb-638f-4a33-ba29-b95acae8086b None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:33.866636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c3ca4ebb-638f-4a33-ba29-b95acae8086b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:33.997353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca4ebb-638f-4a33-ba29-b95acae8086b None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:34.077842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0a29c-3a39-4725-ace1-d3dbbf853ee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:34.078936 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0a29c-3a39-4725-ace1-d3dbbf853ee9 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:34.079419 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0a29c-3a39-4725-ace1-d3dbbf853ee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:34.079894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0a29c-3a39-4725-ace1-d3dbbf853ee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:34.081734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-19a0a29c-3a39-4725-ace1-d3dbbf853ee9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:34.140927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:34.197622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0a29c-3a39-4725-ace1-d3dbbf853ee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:34.199072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0a29c-3a39-4725-ace1-d3dbbf853ee9 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:34.219643 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 661/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:43:36.470583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ab347-9273-49b9-aa39-aeebec435843 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:36.471160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ab347-9273-49b9-aa39-aeebec435843 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:36.471444 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ab347-9273-49b9-aa39-aeebec435843 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:36.473318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 651/1312] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:43:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:43:36.479145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949be29f-1142-4d09-8fde-fb5b4e49cb0e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:36.479624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949be29f-1142-4d09-8fde-fb5b4e49cb0e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:36.479738 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949be29f-1142-4d09-8fde-fb5b4e49cb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:36.564502 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-949be29f-1142-4d09-8fde-fb5b4e49cb0e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:36.565887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-949be29f-1142-4d09-8fde-fb5b4e49cb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:36.670691 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 662/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:36.955950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ab347-9273-49b9-aa39-aeebec435843 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:36.957540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-558ab347-9273-49b9-aa39-aeebec435843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:37.060232 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ab347-9273-49b9-aa39-aeebec435843 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:37.131692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:37.134253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee76aa-09ff-460c-a91d-dffe4fd09906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:37.134873 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee76aa-09ff-460c-a91d-dffe4fd09906 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:37.135154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee76aa-09ff-460c-a91d-dffe4fd09906 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:37.135429 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee76aa-09ff-460c-a91d-dffe4fd09906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:37.137063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-64ee76aa-09ff-460c-a91d-dffe4fd09906 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:37.244613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee76aa-09ff-460c-a91d-dffe4fd09906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:37.246096 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee76aa-09ff-460c-a91d-dffe4fd09906 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:37.264611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 652/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:43:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:43:39.697059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09540d54-db75-4b04-92b1-2c261168ea8c None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:39.697588 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09540d54-db75-4b04-92b1-2c261168ea8c None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:39.697683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09540d54-db75-4b04-92b1-2c261168ea8c None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:39.698956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 663/1315] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:43:39.704948 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df59f4-94e1-493a-a5e8-174970ad594e None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:39.705338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df59f4-94e1-493a-a5e8-174970ad594e None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:39.705496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df59f4-94e1-493a-a5e8-174970ad594e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:39.771479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-58df59f4-94e1-493a-a5e8-174970ad594e None None] MFA Rules not processed for user `54d0a424442447c49a385a116cc9eec2`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:39.772733 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-58df59f4-94e1-493a-a5e8-174970ad594e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:39.823988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:39.868800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 653/1316] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:43:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:40.153004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09540d54-db75-4b04-92b1-2c261168ea8c None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:40.154548 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-09540d54-db75-4b04-92b1-2c261168ea8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:40.164814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:40.256086 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09540d54-db75-4b04-92b1-2c261168ea8c None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:40.326705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61e120-6e86-4a33-9b92-28453b46801d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:40.327328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61e120-6e86-4a33-9b92-28453b46801d service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:40.327693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61e120-6e86-4a33-9b92-28453b46801d service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:40.327989 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61e120-6e86-4a33-9b92-28453b46801d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:40.329604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9d61e120-6e86-4a33-9b92-28453b46801d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:40.390122 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:40.435993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61e120-6e86-4a33-9b92-28453b46801d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:40.437417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61e120-6e86-4a33-9b92-28453b46801d service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:40.456230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 664/1317] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:43:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:43:43.196708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef136d80-8685-4812-9b1c-e99b3c3ca6fa None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:43.197295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef136d80-8685-4812-9b1c-e99b3c3ca6fa None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:43.197295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef136d80-8685-4812-9b1c-e99b3c3ca6fa None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:43.198609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 654/1318] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:43:43.204448 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd9c77-d527-4ef4-a39d-1c2156feb4b1 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:43.204933 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd9c77-d527-4ef4-a39d-1c2156feb4b1 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:43.205088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd9c77-d527-4ef4-a39d-1c2156feb4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:43.269341 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-f3cd9c77-d527-4ef4-a39d-1c2156feb4b1 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:43.270536 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f3cd9c77-d527-4ef4-a39d-1c2156feb4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:43.368985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 665/1319] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:43:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:43:43.656445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef136d80-8685-4812-9b1c-e99b3c3ca6fa None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:43.658008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ef136d80-8685-4812-9b1c-e99b3c3ca6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:43.758876 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef136d80-8685-4812-9b1c-e99b3c3ca6fa None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:43.831578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7294609-e39b-4986-9ca4-9d3d980cdd89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:43.832185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7294609-e39b-4986-9ca4-9d3d980cdd89 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:43.832474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7294609-e39b-4986-9ca4-9d3d980cdd89 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:43.832782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7294609-e39b-4986-9ca4-9d3d980cdd89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:43.834400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b7294609-e39b-4986-9ca4-9d3d980cdd89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:43.842010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:43.942836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7294609-e39b-4986-9ca4-9d3d980cdd89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:43.944310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7294609-e39b-4986-9ca4-9d3d980cdd89 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:43.962804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 655/1320] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:43:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:43:47.096551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a05be-a01f-4864-9ad8-e57e92431648 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:47.097179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a05be-a01f-4864-9ad8-e57e92431648 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:47.097179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a05be-a01f-4864-9ad8-e57e92431648 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:47.098488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 666/1321] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:43:47.103797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f90dd-0544-44db-b3d4-343cc6344a64 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:47.104207 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f90dd-0544-44db-b3d4-343cc6344a64 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:47.104539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f90dd-0544-44db-b3d4-343cc6344a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:47.170862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-463f90dd-0544-44db-b3d4-343cc6344a64 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:47.172214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-463f90dd-0544-44db-b3d4-343cc6344a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:47.231423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:47.270710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 656/1322] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:47.558264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a05be-a01f-4864-9ad8-e57e92431648 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:47.559811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-218a05be-a01f-4864-9ad8-e57e92431648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:47.580617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:47.666555 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a05be-a01f-4864-9ad8-e57e92431648 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:47.740820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b8841-5b6f-40fe-8d8f-086b4b797f4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:47.741429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b8841-5b6f-40fe-8d8f-086b4b797f4f service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:47.741713 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b8841-5b6f-40fe-8d8f-086b4b797f4f service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:47.742011 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b8841-5b6f-40fe-8d8f-086b4b797f4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:47.743617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b64b8841-5b6f-40fe-8d8f-086b4b797f4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:47.815363 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:47.852957 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b8841-5b6f-40fe-8d8f-086b4b797f4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:47.854442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b8841-5b6f-40fe-8d8f-086b4b797f4f service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:47.872460 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 667/1323] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:43:49.862570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a5376-2ebf-43f5-be2f-ffb252eda426 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:49.863251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a5376-2ebf-43f5-be2f-ffb252eda426 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:49.863251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a5376-2ebf-43f5-be2f-ffb252eda426 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:49.864461 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 657/1324] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:43:49.869905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98e91a-748d-45f7-b6a5-6d6ee5ce6fb2 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:49.870384 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98e91a-748d-45f7-b6a5-6d6ee5ce6fb2 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:49.870489 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98e91a-748d-45f7-b6a5-6d6ee5ce6fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:49.937137 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-1c98e91a-748d-45f7-b6a5-6d6ee5ce6fb2 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:49.938521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-1c98e91a-748d-45f7-b6a5-6d6ee5ce6fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:50.033861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 668/1325] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:50.314837 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a5376-2ebf-43f5-be2f-ffb252eda426 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:50.316423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-bf0a5376-2ebf-43f5-be2f-ffb252eda426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:50.420183 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a5376-2ebf-43f5-be2f-ffb252eda426 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:50.503429 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac47106-320b-42eb-9ece-55b2c6ef69d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:50.504040 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac47106-320b-42eb-9ece-55b2c6ef69d0 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:50.504320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac47106-320b-42eb-9ece-55b2c6ef69d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:50.504592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac47106-320b-42eb-9ece-55b2c6ef69d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:50.506271 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5ac47106-320b-42eb-9ece-55b2c6ef69d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:50.525124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:50.626482 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac47106-320b-42eb-9ece-55b2c6ef69d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:50.627927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac47106-320b-42eb-9ece-55b2c6ef69d0 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:50.646804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 658/1326] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:43:52.630050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2d94-ce2a-4232-a15c-57308bc257cc None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:52.630585 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2d94-ce2a-4232-a15c-57308bc257cc None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:52.630686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2d94-ce2a-4232-a15c-57308bc257cc None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:52.632000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 669/1327] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:43:52.637351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37a0a9-f61d-4147-9274-25a5f876a713 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:52.637849 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37a0a9-f61d-4147-9274-25a5f876a713 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:52.637958 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37a0a9-f61d-4147-9274-25a5f876a713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:52.710275 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-0e37a0a9-f61d-4147-9274-25a5f876a713 None None] MFA Rules not processed for user `54d0a424442447c49a385a116cc9eec2`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:52.711766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0e37a0a9-f61d-4147-9274-25a5f876a713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:52.776776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:52.812698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 659/1328] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:43:52 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:53.097924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb2d94-ce2a-4232-a15c-57308bc257cc None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:53.099458 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-fcdb2d94-ce2a-4232-a15c-57308bc257cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:53.136213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:53.214082 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb2d94-ce2a-4232-a15c-57308bc257cc None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:53.291952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678b897-59c4-40b0-ad74-6f42c2b6a860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:53.292999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678b897-59c4-40b0-ad74-6f42c2b6a860 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:53.293468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678b897-59c4-40b0-ad74-6f42c2b6a860 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:53.293923 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678b897-59c4-40b0-ad74-6f42c2b6a860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:53.297361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f678b897-59c4-40b0-ad74-6f42c2b6a860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:53.377071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:53.408044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678b897-59c4-40b0-ad74-6f42c2b6a860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:53.409535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678b897-59c4-40b0-ad74-6f42c2b6a860 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:53.427474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 670/1329] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:43:53 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:43:55.913734 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f6d4b-87a2-466d-993a-26bbfebd25a3 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:55.914315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f6d4b-87a2-466d-993a-26bbfebd25a3 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:55.914315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f6d4b-87a2-466d-993a-26bbfebd25a3 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:55.915647 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 660/1330] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:43:55.920962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83af40-550e-435e-977b-0e9335578a02 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:55.921391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83af40-550e-435e-977b-0e9335578a02 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:55.921524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83af40-550e-435e-977b-0e9335578a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:55.987093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-eb83af40-550e-435e-977b-0e9335578a02 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:55.988293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-eb83af40-550e-435e-977b-0e9335578a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:56.083487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 671/1331] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:56.367124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f6d4b-87a2-466d-993a-26bbfebd25a3 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:56.368696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-dd1f6d4b-87a2-466d-993a-26bbfebd25a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:56.469483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f6d4b-87a2-466d-993a-26bbfebd25a3 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:56.541154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2110388-a7e7-4216-ab7d-3b0bdef3613b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:56.541752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2110388-a7e7-4216-ab7d-3b0bdef3613b service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:56.542030 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2110388-a7e7-4216-ab7d-3b0bdef3613b service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:56.542301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2110388-a7e7-4216-ab7d-3b0bdef3613b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:56.543890 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b2110388-a7e7-4216-ab7d-3b0bdef3613b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:56.570642 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:56.650898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2110388-a7e7-4216-ab7d-3b0bdef3613b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:56.652257 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2110388-a7e7-4216-ab7d-3b0bdef3613b service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:56.670589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 661/1332] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:43:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:43:58.648189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1dff2-9198-4132-95de-377e8d399cb0 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:58.648774 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1dff2-9198-4132-95de-377e8d399cb0 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:58.648774 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1dff2-9198-4132-95de-377e8d399cb0 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:58.650135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 672/1333] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:43:58.655470 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d84c2a-5ee6-4051-ba9c-e9f35009a382 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:58.655874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d84c2a-5ee6-4051-ba9c-e9f35009a382 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:58.656023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d84c2a-5ee6-4051-ba9c-e9f35009a382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:58.721112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-07d84c2a-5ee6-4051-ba9c-e9f35009a382 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:43:58.722294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-07d84c2a-5ee6-4051-ba9c-e9f35009a382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:58.795877 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:58.818656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 662/1334] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:43:59.105302 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1dff2-9198-4132-95de-377e8d399cb0 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:43:59.106846 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-98d1dff2-9198-4132-95de-377e8d399cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:59.147009 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:59.214127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1dff2-9198-4132-95de-377e8d399cb0 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:43:59.290848 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45daa88-e312-44e3-9e3f-5b04c890faa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:43:59.291446 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45daa88-e312-44e3-9e3f-5b04c890faa3 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:43:59.291725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45daa88-e312-44e3-9e3f-5b04c890faa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:43:59.291996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45daa88-e312-44e3-9e3f-5b04c890faa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:43:59.293608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c45daa88-e312-44e3-9e3f-5b04c890faa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:43:59.377906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:43:59.403239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45daa88-e312-44e3-9e3f-5b04c890faa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:43:59.404750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45daa88-e312-44e3-9e3f-5b04c890faa3 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:43:59.424236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 673/1335] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:43:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:01.957252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cba7a-130f-44b7-94a2-29a363b33341 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:01.957813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cba7a-130f-44b7-94a2-29a363b33341 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:01.957813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cba7a-130f-44b7-94a2-29a363b33341 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:01.959181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 663/1336] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:01.964876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c938e-84df-48b4-8b40-34cb3636ce86 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:01.965313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c938e-84df-48b4-8b40-34cb3636ce86 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:01.965429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c938e-84df-48b4-8b40-34cb3636ce86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:02.030540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-285c938e-84df-48b4-8b40-34cb3636ce86 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:02.031746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-285c938e-84df-48b4-8b40-34cb3636ce86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:02.126614 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 674/1337] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:02.420063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe0a76-ac28-4984-8900-c592143c220c None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:02.420346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe0a76-ac28-4984-8900-c592143c220c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:02.420620 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe0a76-ac28-4984-8900-c592143c220c None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:02.421992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 664/1338] 10.222.0.26 () {54 vars in 965 bytes} [Wed Aug 24 07:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:02.429497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc92d1-a25f-4b67-9444-cc777da90a90 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:02.429790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc92d1-a25f-4b67-9444-cc777da90a90 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:02.430070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc92d1-a25f-4b67-9444-cc777da90a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:02.483612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:02.506105 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-8bdc92d1-a25f-4b67-9444-cc777da90a90 None None] MFA Rules not processed for user `46e61402c01c4f9ba5f18e94960bc313`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:02.507336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8bdc92d1-a25f-4b67-9444-cc777da90a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:02.585741 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 675/1339] 10.222.0.26 () {58 vars in 1062 bytes} [Wed Aug 24 07:44:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:02.595428 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe0a76-ac28-4984-8900-c592143c220c None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:02.597151 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-96fe0a76-ac28-4984-8900-c592143c220c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:02.699627 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe0a76-ac28-4984-8900-c592143c220c None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:02.771732 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ca83d-c939-49a4-af7c-1aaa38b3e387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:02.772327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ca83d-c939-49a4-af7c-1aaa38b3e387 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:02.772607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ca83d-c939-49a4-af7c-1aaa38b3e387 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:02.772909 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ca83d-c939-49a4-af7c-1aaa38b3e387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:02.774497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6a4ca83d-c939-49a4-af7c-1aaa38b3e387 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:02.811057 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:02.884521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ca83d-c939-49a4-af7c-1aaa38b3e387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:02.886355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ca83d-c939-49a4-af7c-1aaa38b3e387 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:02.905261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 665/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:07.567262 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70b39b-45c3-4bba-885f-dec919ea2fbc None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:07.567827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70b39b-45c3-4bba-885f-dec919ea2fbc None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:07.567827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70b39b-45c3-4bba-885f-dec919ea2fbc None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:07.569207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 676/1341] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:07.574565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab84ea-3801-4658-bfc8-f50157f58c5c None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:07.575021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab84ea-3801-4658-bfc8-f50157f58c5c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:07.575126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab84ea-3801-4658-bfc8-f50157f58c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:07.643458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-7aab84ea-3801-4658-bfc8-f50157f58c5c None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:07.644688 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-7aab84ea-3801-4658-bfc8-f50157f58c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:07.729313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:07.740982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 666/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:08.023994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70b39b-45c3-4bba-885f-dec919ea2fbc None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:08.025576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6f70b39b-45c3-4bba-885f-dec919ea2fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:08.129452 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70b39b-45c3-4bba-885f-dec919ea2fbc None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:08.167293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:08.205332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c8497-a883-4c1d-9f6f-24da45baf349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:08.205936 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c8497-a883-4c1d-9f6f-24da45baf349 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:08.206217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c8497-a883-4c1d-9f6f-24da45baf349 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:08.206488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c8497-a883-4c1d-9f6f-24da45baf349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:08.208082 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6e6c8497-a883-4c1d-9f6f-24da45baf349 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:08.320422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c8497-a883-4c1d-9f6f-24da45baf349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:08.321914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c8497-a883-4c1d-9f6f-24da45baf349 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:08.341122 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 677/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:11.181938 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84d83a-e87b-421d-87b1-948689376c0b None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:11.182498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84d83a-e87b-421d-87b1-948689376c0b None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:11.182498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84d83a-e87b-421d-87b1-948689376c0b None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:11.183994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 667/1344] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:11.189039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ffaaa-9d2b-46ab-a099-b9b3da8a3646 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:11.189456 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ffaaa-9d2b-46ab-a099-b9b3da8a3646 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:11.189706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ffaaa-9d2b-46ab-a099-b9b3da8a3646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:11.256658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9f5ffaaa-9d2b-46ab-a099-b9b3da8a3646 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:11.257794 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9f5ffaaa-9d2b-46ab-a099-b9b3da8a3646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:11.275823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:11.355448 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 678/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:11.639232 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84d83a-e87b-421d-87b1-948689376c0b None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:11.640832 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2d84d83a-e87b-421d-87b1-948689376c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:11.752099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84d83a-e87b-421d-87b1-948689376c0b None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:11.827222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032591b3-0c5f-43f4-a696-ddd3fab0e145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:11.827825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032591b3-0c5f-43f4-a696-ddd3fab0e145 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:11.828106 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032591b3-0c5f-43f4-a696-ddd3fab0e145 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:11.828378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032591b3-0c5f-43f4-a696-ddd3fab0e145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:11.830046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-032591b3-0c5f-43f4-a696-ddd3fab0e145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:11.875396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:11.944874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032591b3-0c5f-43f4-a696-ddd3fab0e145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:11.946387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032591b3-0c5f-43f4-a696-ddd3fab0e145 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:11.966556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 668/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:16.424362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005dac36-d5f2-4856-9c5a-9e6df421dd70 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:16.424962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005dac36-d5f2-4856-9c5a-9e6df421dd70 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:16.424962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005dac36-d5f2-4856-9c5a-9e6df421dd70 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:16.426315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 679/1347] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:16.431792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc48264-181e-42f0-b98b-1abde9c47845 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:16.432287 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc48264-181e-42f0-b98b-1abde9c47845 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:16.432383 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc48264-181e-42f0-b98b-1abde9c47845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:16.502876 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-9fc48264-181e-42f0-b98b-1abde9c47845 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:16.504129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9fc48264-181e-42f0-b98b-1abde9c47845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:16.603974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:16.606878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 669/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:16.893715 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005dac36-d5f2-4856-9c5a-9e6df421dd70 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:16.895278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-005dac36-d5f2-4856-9c5a-9e6df421dd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:17.013603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005dac36-d5f2-4856-9c5a-9e6df421dd70 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:17.066295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:17.092396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab9fee-4f18-4d91-8d35-42c6cc8a2b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:17.093035 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab9fee-4f18-4d91-8d35-42c6cc8a2b13 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:17.093314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab9fee-4f18-4d91-8d35-42c6cc8a2b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:17.093582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab9fee-4f18-4d91-8d35-42c6cc8a2b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:17.095276 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-62ab9fee-4f18-4d91-8d35-42c6cc8a2b13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:17.206747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab9fee-4f18-4d91-8d35-42c6cc8a2b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:17.208198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab9fee-4f18-4d91-8d35-42c6cc8a2b13 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:17.226664 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 680/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:22.144875 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d74258-4dd1-4116-ba7d-7af22208f6c6 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:22.145458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d74258-4dd1-4116-ba7d-7af22208f6c6 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:22.145458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d74258-4dd1-4116-ba7d-7af22208f6c6 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:22.147407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 670/1350] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 24 07:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:22.152205 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac8ebe-de35-4752-9813-558d4be31f02 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:22.152675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac8ebe-de35-4752-9813-558d4be31f02 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:22.152776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac8ebe-de35-4752-9813-558d4be31f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:22.254677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-f6ac8ebe-de35-4752-9813-558d4be31f02 None None] MFA Rules not processed for user `ea3816d5ae054741ac2c5a2eec3eb617`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:22.255884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f6ac8ebe-de35-4752-9813-558d4be31f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:22.285810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:22.440151 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 681/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 24 07:44:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:26.744232 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69353e8f-67bf-40cf-a789-a6227fbdff88 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:26.744739 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69353e8f-67bf-40cf-a789-a6227fbdff88 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:26.744739 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69353e8f-67bf-40cf-a789-a6227fbdff88 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:26.746219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 671/1352] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 24 07:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:26.751255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8c072-c56f-4013-b0b7-df096f3f8929 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:26.752634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8c072-c56f-4013-b0b7-df096f3f8929 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:26.753167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8c072-c56f-4013-b0b7-df096f3f8929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:26.835133 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-39d8c072-c56f-4013-b0b7-df096f3f8929 None None] MFA Rules not processed for user `ea3816d5ae054741ac2c5a2eec3eb617`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:26.836795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-39d8c072-c56f-4013-b0b7-df096f3f8929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:26.912751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:26.950079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 682/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 24 07:44:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:28.251732 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafeeaf-6a51-4b0a-9150-9f36cf3608f1 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:28.252318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafeeaf-6a51-4b0a-9150-9f36cf3608f1 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:28.252318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafeeaf-6a51-4b0a-9150-9f36cf3608f1 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:28.253974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 672/1354] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 24 07:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:28.259050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809af859-c290-42ab-a529-69af9ee93f21 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:28.259508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809af859-c290-42ab-a529-69af9ee93f21 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:28.259661 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809af859-c290-42ab-a529-69af9ee93f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:28.346072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-809af859-c290-42ab-a529-69af9ee93f21 None None] MFA Rules not processed for user `ea3816d5ae054741ac2c5a2eec3eb617`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:28.347318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-809af859-c290-42ab-a529-69af9ee93f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:28.457780 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 683/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 24 07:44:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:31.910360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6563120-e60b-4934-b99d-57822a2f97f5 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:31.910949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6563120-e60b-4934-b99d-57822a2f97f5 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:31.910949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6563120-e60b-4934-b99d-57822a2f97f5 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:31.912222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 673/1356] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:31.921835 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536c127-acb4-49c8-8c5d-4f2167f25261 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:31.922259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536c127-acb4-49c8-8c5d-4f2167f25261 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:31.922368 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536c127-acb4-49c8-8c5d-4f2167f25261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:31.958910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:32.013439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-7536c127-acb4-49c8-8c5d-4f2167f25261 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:32.016482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7536c127-acb4-49c8-8c5d-4f2167f25261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:32.137147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 684/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:32.442162 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0815ee7-f976-4178-ae81-4b74432ca4cf None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:32.442641 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0815ee7-f976-4178-ae81-4b74432ca4cf None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:32.443126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0815ee7-f976-4178-ae81-4b74432ca4cf None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:32.445042 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 674/1358] 10.222.0.26 () {54 vars in 962 bytes} [Wed Aug 24 07:44:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:32.454716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1c420-930e-4f98-bde6-b7d6c1675878 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:32.455083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1c420-930e-4f98-bde6-b7d6c1675878 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:32.455518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1c420-930e-4f98-bde6-b7d6c1675878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:32.554835 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-eda1c420-930e-4f98-bde6-b7d6c1675878 None None] MFA Rules not processed for user `bdf81f09006a42299c0807f77f16c748`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:32.556930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-eda1c420-930e-4f98-bde6-b7d6c1675878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:32.616457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:32.695111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 685/1359] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Aug 24 07:44:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:32.703830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0815ee7-f976-4178-ae81-4b74432ca4cf None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:32.705571 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-a0815ee7-f976-4178-ae81-4b74432ca4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:32.867324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0815ee7-f976-4178-ae81-4b74432ca4cf None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:32.949044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:32.993271 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8176fd-6442-461f-a4e1-645927f0e59a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:32.993930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8176fd-6442-461f-a4e1-645927f0e59a service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:32.994214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8176fd-6442-461f-a4e1-645927f0e59a service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:32.994487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8176fd-6442-461f-a4e1-645927f0e59a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:32.996081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-3d8176fd-6442-461f-a4e1-645927f0e59a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:33.093419 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8176fd-6442-461f-a4e1-645927f0e59a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:33.095560 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8176fd-6442-461f-a4e1-645927f0e59a service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:33.115863 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 675/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:35.283786 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b1f70-6576-437f-8341-5bdcfd7c892a None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:35.284349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b1f70-6576-437f-8341-5bdcfd7c892a None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:35.284349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b1f70-6576-437f-8341-5bdcfd7c892a None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:35.285902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 686/1361] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:35.291069 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d124f19-d8c3-4126-96b4-afb36ee48cc4 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:35.291502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d124f19-d8c3-4126-96b4-afb36ee48cc4 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:35.291673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d124f19-d8c3-4126-96b4-afb36ee48cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:35.363952 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-3d124f19-d8c3-4126-96b4-afb36ee48cc4 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:35.365333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-3d124f19-d8c3-4126-96b4-afb36ee48cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:35.410526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:35.486922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 676/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:35.788425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b1f70-6576-437f-8341-5bdcfd7c892a None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:35.790079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f96b1f70-6576-437f-8341-5bdcfd7c892a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:35.947325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b1f70-6576-437f-8341-5bdcfd7c892a None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:35.957261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:36.071707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278eef7-b965-4f82-a1d9-773ea8231f12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:36.072307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278eef7-b965-4f82-a1d9-773ea8231f12 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:36.072607 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278eef7-b965-4f82-a1d9-773ea8231f12 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:36.072906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278eef7-b965-4f82-a1d9-773ea8231f12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:36.074515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6278eef7-b965-4f82-a1d9-773ea8231f12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:36.170839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278eef7-b965-4f82-a1d9-773ea8231f12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:36.172433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278eef7-b965-4f82-a1d9-773ea8231f12 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:36.193626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 687/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:38.598272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb92e9-49ec-42af-950b-a7ec439579ae None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:38.599146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb92e9-49ec-42af-950b-a7ec439579ae None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:38.599607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb92e9-49ec-42af-950b-a7ec439579ae None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:38.600987 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 677/1364] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:38.606477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8ee85-1298-4764-a0bc-192dd2c225d3 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:38.606914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8ee85-1298-4764-a0bc-192dd2c225d3 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:38.607081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8ee85-1298-4764-a0bc-192dd2c225d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:38.623694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:38.676531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-f0e8ee85-1298-4764-a0bc-192dd2c225d3 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:38.677977 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f0e8ee85-1298-4764-a0bc-192dd2c225d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:38.783717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 688/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:39.090670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ea8b3-75ed-4766-b0f4-0051949072b8 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:39.091967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ea8b3-75ed-4766-b0f4-0051949072b8 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:39.092434 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ea8b3-75ed-4766-b0f4-0051949072b8 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:39.094007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 678/1366] 10.222.0.26 () {54 vars in 962 bytes} [Wed Aug 24 07:44:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:39.099921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885ab59-26a2-4460-aa0c-174564923a2f None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:39.104437 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885ab59-26a2-4460-aa0c-174564923a2f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:39.104437 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885ab59-26a2-4460-aa0c-174564923a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:39.184791 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9885ab59-26a2-4460-aa0c-174564923a2f None None] MFA Rules not processed for user `bdf81f09006a42299c0807f77f16c748`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:39.185990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9885ab59-26a2-4460-aa0c-174564923a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:39.224952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:39.317056 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 689/1367] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Aug 24 07:44:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:39.324893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ea8b3-75ed-4766-b0f4-0051949072b8 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:39.326501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fa2ea8b3-75ed-4766-b0f4-0051949072b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:39.479698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ea8b3-75ed-4766-b0f4-0051949072b8 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:39.501153 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:39.616715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa2b56-745f-4a53-86b1-f95d71a0bebc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:39.617348 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa2b56-745f-4a53-86b1-f95d71a0bebc service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:39.617579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa2b56-745f-4a53-86b1-f95d71a0bebc service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:39.617787 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa2b56-745f-4a53-86b1-f95d71a0bebc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:39.619346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d1aa2b56-745f-4a53-86b1-f95d71a0bebc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:39.712107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa2b56-745f-4a53-86b1-f95d71a0bebc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:39.714124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa2b56-745f-4a53-86b1-f95d71a0bebc service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:39.733726 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 679/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:40.152399 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27dfcb-9260-4e54-bb16-9680b307b993 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:40.152953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27dfcb-9260-4e54-bb16-9680b307b993 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:40.152953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27dfcb-9260-4e54-bb16-9680b307b993 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:40.154474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 690/1369] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:44:40.160592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5ecfc-e150-4ca5-8a6b-f0d79ca51229 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:40.160914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5ecfc-e150-4ca5-8a6b-f0d79ca51229 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:40.161197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5ecfc-e150-4ca5-8a6b-f0d79ca51229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:40.194592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:40.232422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d4b5ecfc-e150-4ca5-8a6b-f0d79ca51229 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:40.234604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d4b5ecfc-e150-4ca5-8a6b-f0d79ca51229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:40.338244 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 680/1370] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:40.925486 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042befbb-0e60-42f9-b9f4-d24501e44a95 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:40.925486 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042befbb-0e60-42f9-b9f4-d24501e44a95 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:40.926902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042befbb-0e60-42f9-b9f4-d24501e44a95 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:40.928685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 691/1371] 10.222.0.26 () {54 vars in 964 bytes} [Wed Aug 24 07:44:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:40.932808 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9838d5-286e-40bb-8db9-c4f4e93b169e None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:40.933357 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9838d5-286e-40bb-8db9-c4f4e93b169e None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:40.933629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9838d5-286e-40bb-8db9-c4f4e93b169e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:41.018936 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-9e9838d5-286e-40bb-8db9-c4f4e93b169e None None] MFA Rules not processed for user `43e57131a5be4af785166bc03a3337a2`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:41.020253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9e9838d5-286e-40bb-8db9-c4f4e93b169e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:41.077760 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:41.108179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 681/1372] 10.222.0.26 () {58 vars in 1061 bytes} [Wed Aug 24 07:44:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:41.116688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042befbb-0e60-42f9-b9f4-d24501e44a95 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:41.118208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-042befbb-0e60-42f9-b9f4-d24501e44a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:41.226971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042befbb-0e60-42f9-b9f4-d24501e44a95 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:41.271331 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:41.308886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a11c73-b9b1-49ce-b1c6-ca6c9be4e1e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:41.309560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a11c73-b9b1-49ce-b1c6-ca6c9be4e1e4 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:41.309852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a11c73-b9b1-49ce-b1c6-ca6c9be4e1e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:41.310125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a11c73-b9b1-49ce-b1c6-ca6c9be4e1e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:41.311802 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c0a11c73-b9b1-49ce-b1c6-ca6c9be4e1e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:41.424952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a11c73-b9b1-49ce-b1c6-ca6c9be4e1e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:41.426495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a11c73-b9b1-49ce-b1c6-ca6c9be4e1e4 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:41.446236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 692/1373] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:41.914088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed578fc-8bbb-45a0-82cc-1e82a8727c6d None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:41.914468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed578fc-8bbb-45a0-82cc-1e82a8727c6d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:41.914929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed578fc-8bbb-45a0-82cc-1e82a8727c6d None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:41.916427 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 682/1374] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:41.921857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd43667-5845-4b73-8cd0-d67882713985 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:41.922154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd43667-5845-4b73-8cd0-d67882713985 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:41.922435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd43667-5845-4b73-8cd0-d67882713985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:41.989652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-8fd43667-5845-4b73-8cd0-d67882713985 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:41.990921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8fd43667-5845-4b73-8cd0-d67882713985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:42.011640 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:42.088666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 693/1375] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:42.379942 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed578fc-8bbb-45a0-82cc-1e82a8727c6d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:42.381829 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-1ed578fc-8bbb-45a0-82cc-1e82a8727c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:42.528936 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed578fc-8bbb-45a0-82cc-1e82a8727c6d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:42.583423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:42.652138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d041858-98b5-4cc2-bdd8-b74598017ba2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:42.652773 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d041858-98b5-4cc2-bdd8-b74598017ba2 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:42.653052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d041858-98b5-4cc2-bdd8-b74598017ba2 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:42.653324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d041858-98b5-4cc2-bdd8-b74598017ba2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:42.654916 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4d041858-98b5-4cc2-bdd8-b74598017ba2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:42.745984 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d041858-98b5-4cc2-bdd8-b74598017ba2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:42.747567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d041858-98b5-4cc2-bdd8-b74598017ba2 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:42.764975 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 683/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:44.879159 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dac41c-7f16-4b2a-9bf3-20d822484ccc None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:44.879635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dac41c-7f16-4b2a-9bf3-20d822484ccc None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:44.879726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dac41c-7f16-4b2a-9bf3-20d822484ccc None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:44.881108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 694/1377] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:44.886463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ee3cd-a7f3-437f-878f-2051586cd1d0 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:44.886909 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ee3cd-a7f3-437f-878f-2051586cd1d0 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:44.887046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ee3cd-a7f3-437f-878f-2051586cd1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:44.962193 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-cb0ee3cd-a7f3-437f-878f-2051586cd1d0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:44.963417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-cb0ee3cd-a7f3-437f-878f-2051586cd1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:44.970244 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:45.076934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 684/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:45.381929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dac41c-7f16-4b2a-9bf3-20d822484ccc None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:45.383481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-61dac41c-7f16-4b2a-9bf3-20d822484ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:45.543264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dac41c-7f16-4b2a-9bf3-20d822484ccc None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:45.547370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:45.668171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f40e10-96ee-4cf4-aa92-0cd64fb3e262 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:45.668723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f40e10-96ee-4cf4-aa92-0cd64fb3e262 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:45.669007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f40e10-96ee-4cf4-aa92-0cd64fb3e262 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:45.669283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f40e10-96ee-4cf4-aa92-0cd64fb3e262 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:45.670875 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-68f40e10-96ee-4cf4-aa92-0cd64fb3e262 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:45.765852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f40e10-96ee-4cf4-aa92-0cd64fb3e262 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:45.767985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f40e10-96ee-4cf4-aa92-0cd64fb3e262 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:45.786769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 695/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:47.968073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b7c43-b92f-455a-b346-6df34fc7402d None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:47.968073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b7c43-b92f-455a-b346-6df34fc7402d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:47.968073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b7c43-b92f-455a-b346-6df34fc7402d None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:47.968073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 685/1380] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:44:47.973764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547704d-b4a0-4e7f-9ec4-f14fc41411f6 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:47.974178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547704d-b4a0-4e7f-9ec4-f14fc41411f6 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:47.974396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547704d-b4a0-4e7f-9ec4-f14fc41411f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:47.986723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:48.046687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a547704d-b4a0-4e7f-9ec4-f14fc41411f6 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:48.048517 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a547704d-b4a0-4e7f-9ec4-f14fc41411f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:48.168812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 696/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:48.462198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b7c43-b92f-455a-b346-6df34fc7402d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:48.463765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-250b7c43-b92f-455a-b346-6df34fc7402d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:48.618462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:48.629203 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b7c43-b92f-455a-b346-6df34fc7402d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:48.747533 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e08ef-6c0c-42d2-baec-1c18860c8463 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:48.748139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e08ef-6c0c-42d2-baec-1c18860c8463 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:48.748415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e08ef-6c0c-42d2-baec-1c18860c8463 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:48.748709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e08ef-6c0c-42d2-baec-1c18860c8463 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:48.750280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-518e08ef-6c0c-42d2-baec-1c18860c8463 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:48.843568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e08ef-6c0c-42d2-baec-1c18860c8463 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:48.845862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e08ef-6c0c-42d2-baec-1c18860c8463 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:48.856715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:48.866521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 686/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:51.232227 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ac717-6330-4dd9-a236-db54794e90a9 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:51.232819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ac717-6330-4dd9-a236-db54794e90a9 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:51.232939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ac717-6330-4dd9-a236-db54794e90a9 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:51.234627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 697/1383] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:51.241312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c0c81-03f7-40fe-827a-089ad8c266fe None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:51.241788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c0c81-03f7-40fe-827a-089ad8c266fe None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:51.241891 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c0c81-03f7-40fe-827a-089ad8c266fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:51.311518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-cc5c0c81-03f7-40fe-827a-089ad8c266fe None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:51.312803 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-cc5c0c81-03f7-40fe-827a-089ad8c266fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:51.415644 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 687/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:51.708591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ac717-6330-4dd9-a236-db54794e90a9 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:51.710545 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-614ac717-6330-4dd9-a236-db54794e90a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:51.800616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:51.867175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ac717-6330-4dd9-a236-db54794e90a9 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:51.986960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc508d7-1469-47ad-bf59-1121fc8ce4d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:51.987602 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc508d7-1469-47ad-bf59-1121fc8ce4d2 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:51.987882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc508d7-1469-47ad-bf59-1121fc8ce4d2 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:51.988157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc508d7-1469-47ad-bf59-1121fc8ce4d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:51.989841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3cc508d7-1469-47ad-bf59-1121fc8ce4d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:52.041484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:52.086969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc508d7-1469-47ad-bf59-1121fc8ce4d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:52.088622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc508d7-1469-47ad-bf59-1121fc8ce4d2 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:52.107435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 698/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:54.033907 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6baec-50fe-4c77-8517-8f3c702eb53a None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:54.034468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6baec-50fe-4c77-8517-8f3c702eb53a None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:54.034468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6baec-50fe-4c77-8517-8f3c702eb53a None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:54.036028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 688/1386] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:54.041942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5db73-3069-4659-85d9-9e8517481b15 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:54.042444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5db73-3069-4659-85d9-9e8517481b15 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:54.042557 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5db73-3069-4659-85d9-9e8517481b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:54.116076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-daa5db73-3069-4659-85d9-9e8517481b15 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:54.117323 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-daa5db73-3069-4659-85d9-9e8517481b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:54.218566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 699/1387] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:54.519531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6baec-50fe-4c77-8517-8f3c702eb53a None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:54.521162 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-58b6baec-50fe-4c77-8517-8f3c702eb53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:54.604847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:54.682220 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6baec-50fe-4c77-8517-8f3c702eb53a None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:54.806283 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62726b6-eca7-4886-aae6-2b869d40588e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:54.806909 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62726b6-eca7-4886-aae6-2b869d40588e service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:54.807193 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62726b6-eca7-4886-aae6-2b869d40588e service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:54.807475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62726b6-eca7-4886-aae6-2b869d40588e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:54.809179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f62726b6-eca7-4886-aae6-2b869d40588e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:54.849706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:54.904374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62726b6-eca7-4886-aae6-2b869d40588e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:54.906075 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62726b6-eca7-4886-aae6-2b869d40588e service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:54.925132 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 689/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:44:56.984030 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1565d-6f3c-476d-b54c-4ea057e99520 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:56.984575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1565d-6f3c-476d-b54c-4ea057e99520 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:56.984575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1565d-6f3c-476d-b54c-4ea057e99520 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:56.986633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 700/1389] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:56.992042 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efcc60-ec29-47e3-b3db-4e017afca0cd None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:56.992511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efcc60-ec29-47e3-b3db-4e017afca0cd None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:56.992622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efcc60-ec29-47e3-b3db-4e017afca0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:57.042992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabe15d-e0cb-4663-a67c-558615d04d86 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:57.043415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabe15d-e0cb-4663-a67c-558615d04d86 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:57.043692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabe15d-e0cb-4663-a67c-558615d04d86 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:57.046778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 701/1390] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:44:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:57.054140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee6d6f-0aee-4228-8635-bc12a1f388e1 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:57.054526 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee6d6f-0aee-4228-8635-bc12a1f388e1 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:57.054962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee6d6f-0aee-4228-8635-bc12a1f388e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:57.066245 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-17efcc60-ec29-47e3-b3db-4e017afca0cd None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:57.067509 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-17efcc60-ec29-47e3-b3db-4e017afca0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:57.096928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:57.132814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-e4ee6d6f-0aee-4228-8635-bc12a1f388e1 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:57.134049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e4ee6d6f-0aee-4228-8635-bc12a1f388e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:57.179516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 690/1391] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:57.235739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 702/1392] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:44:57.482998 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247076ac-b92b-4743-b918-775c325d892f None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:57.483388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247076ac-b92b-4743-b918-775c325d892f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:57.483751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247076ac-b92b-4743-b918-775c325d892f None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:57.485337 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 691/1393] 10.222.0.26 () {54 vars in 967 bytes} [Wed Aug 24 07:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:57.490346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4caad29-0a4e-4220-9dcb-3a8b0f031d5a None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:57.490638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4caad29-0a4e-4220-9dcb-3a8b0f031d5a None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:57.490920 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4caad29-0a4e-4220-9dcb-3a8b0f031d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:57.534456 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247076ac-b92b-4743-b918-775c325d892f None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:57.535981 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-247076ac-b92b-4743-b918-775c325d892f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:57.549511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:57.571047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a4caad29-0a4e-4220-9dcb-3a8b0f031d5a None None] MFA Rules not processed for user `38de805ba5eb4aa58cf95d2082c2d216`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:57.572279 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a4caad29-0a4e-4220-9dcb-3a8b0f031d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:57.623636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:57.654476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 703/1394] 10.222.0.26 () {58 vars in 1064 bytes} [Wed Aug 24 07:44:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:57.662656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4caad29-0a4e-4220-9dcb-3a8b0f031d5a None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:57.669913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4caad29-0a4e-4220-9dcb-3a8b0f031d5a None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:57.686664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247076ac-b92b-4743-b918-775c325d892f None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:57.752212 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883d01d-755d-4098-92d9-c1f74a1d28b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38de805ba5eb4aa58cf95d2082c2d216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:57.752770 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883d01d-755d-4098-92d9-c1f74a1d28b0 service glance] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:57.753057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883d01d-755d-4098-92d9-c1f74a1d28b0 service glance] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:57.753330 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883d01d-755d-4098-92d9-c1f74a1d28b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:57.755066 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b883d01d-755d-4098-92d9-c1f74a1d28b0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:57.775574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:57.805118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ca0c5-9dfb-40dc-a25a-b35b9f2d9e5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:57.805809 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ca0c5-9dfb-40dc-a25a-b35b9f2d9e5a service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:57.806101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ca0c5-9dfb-40dc-a25a-b35b9f2d9e5a service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:57.806679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ca0c5-9dfb-40dc-a25a-b35b9f2d9e5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:57.808345 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-188ca0c5-9dfb-40dc-a25a-b35b9f2d9e5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:57.866604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883d01d-755d-4098-92d9-c1f74a1d28b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:57.868330 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883d01d-755d-4098-92d9-c1f74a1d28b0 service glance] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:57.883324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:57.887874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 704/1395] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:44:57.899766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ca0c5-9dfb-40dc-a25a-b35b9f2d9e5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:57.901441 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ca0c5-9dfb-40dc-a25a-b35b9f2d9e5a service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:57.919609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 692/1396] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:44:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:44:58.798317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dbbbf-2ab2-4bfd-a2ca-bd405677fcb2 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:58.798848 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dbbbf-2ab2-4bfd-a2ca-bd405677fcb2 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:58.798848 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dbbbf-2ab2-4bfd-a2ca-bd405677fcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:58.866994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-647dbbbf-2ab2-4bfd-a2ca-bd405677fcb2 None None] MFA Rules not processed for user `be379e1a6add42b69e9fdd676efca280`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:58.868185 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-647dbbbf-2ab2-4bfd-a2ca-bd405677fcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:58.953201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 705/1397] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 24 07:44:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:59.001403 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a8ac7-03ed-4c2a-b802-5fe17339f9b3 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:59.001867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a8ac7-03ed-4c2a-b802-5fe17339f9b3 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:59.002085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a8ac7-03ed-4c2a-b802-5fe17339f9b3 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:59.003544 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 693/1398] 10.222.0.26 () {54 vars in 943 bytes} [Wed Aug 24 07:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:44:59.009742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409745a-8f14-45bd-aff3-30759e58fda7 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:59.010103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409745a-8f14-45bd-aff3-30759e58fda7 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:59.010490 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409745a-8f14-45bd-aff3-30759e58fda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:59.088711 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9409745a-8f14-45bd-aff3-30759e58fda7 None None] MFA Rules not processed for user `b629ad3e6b9f4723b103f9845df2aaaf`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:44:59.089914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9409745a-8f14-45bd-aff3-30759e58fda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:59.122067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:59.201718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 706/1399] 10.222.0.26 () {58 vars in 1040 bytes} [Wed Aug 24 07:44:59 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:44:59.212168 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a8ac7-03ed-4c2a-b802-5fe17339f9b3 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:44:59.213728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b68a8ac7-03ed-4c2a-b802-5fe17339f9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:59.309205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:59.351521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a8ac7-03ed-4c2a-b802-5fe17339f9b3 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:44:59.452828 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8ed5a-077d-4ab4-8c37-8d73365d47f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b629ad3e6b9f4723b103f9845df2aaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:44:59.453999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8ed5a-077d-4ab4-8c37-8d73365d47f0 service swift] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:44:59.454398 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8ed5a-077d-4ab4-8c37-8d73365d47f0 service swift] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:44:59.454677 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8ed5a-077d-4ab4-8c37-8d73365d47f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:44:59.456425 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-00c8ed5a-077d-4ab4-8c37-8d73365d47f0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:44:59.539830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8ed5a-077d-4ab4-8c37-8d73365d47f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:44:59.541567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8ed5a-077d-4ab4-8c37-8d73365d47f0 service swift] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:44:59.545014 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:44:59.549840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 694/1400] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 24 07:44:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:45:00.048841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16d441-1d6f-49f4-9b2a-046bfea15a69 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:00.049484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16d441-1d6f-49f4-9b2a-046bfea15a69 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:00.049484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16d441-1d6f-49f4-9b2a-046bfea15a69 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:00.050788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 707/1401] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:00.057029 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc647c-155c-42da-bb84-d45ee20832ee None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:00.057551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc647c-155c-42da-bb84-d45ee20832ee None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:00.057551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc647c-155c-42da-bb84-d45ee20832ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:00.130440 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-b7dc647c-155c-42da-bb84-d45ee20832ee None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:00.131961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b7dc647c-155c-42da-bb84-d45ee20832ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:00.244730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 695/1402] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:45:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:00.537392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16d441-1d6f-49f4-9b2a-046bfea15a69 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:00.538969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0e16d441-1d6f-49f4-9b2a-046bfea15a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:00.716467 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16d441-1d6f-49f4-9b2a-046bfea15a69 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:00.720207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:00.867961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54bad9-6f24-41ce-aa53-5df373e0ff6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:00.869163 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54bad9-6f24-41ce-aa53-5df373e0ff6b service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:00.869488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54bad9-6f24-41ce-aa53-5df373e0ff6b service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:00.870070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54bad9-6f24-41ce-aa53-5df373e0ff6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:00.871710 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4e54bad9-6f24-41ce-aa53-5df373e0ff6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:00.983359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54bad9-6f24-41ce-aa53-5df373e0ff6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:00.985263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54bad9-6f24-41ce-aa53-5df373e0ff6b service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:01.007421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 708/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:03.644427 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a83b7-853f-421b-8cd9-db229c15aa62 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:03.645466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a83b7-853f-421b-8cd9-db229c15aa62 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:03.645799 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a83b7-853f-421b-8cd9-db229c15aa62 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:03.647376 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 696/1404] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:45:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:03.652937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90b915-0ad8-4449-b0a2-8bb2473fb182 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:03.653797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90b915-0ad8-4449-b0a2-8bb2473fb182 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:03.654262 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90b915-0ad8-4449-b0a2-8bb2473fb182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:03.666614 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:03.731195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-7a90b915-0ad8-4449-b0a2-8bb2473fb182 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:03.732479 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7a90b915-0ad8-4449-b0a2-8bb2473fb182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:03.848686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 709/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:04.139040 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a83b7-853f-421b-8cd9-db229c15aa62 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:04.140565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e85a83b7-853f-421b-8cd9-db229c15aa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:04.233077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:04.307701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a83b7-853f-421b-8cd9-db229c15aa62 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:04.435314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088dba5-93d3-426d-bbee-34689fd85ba3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:04.435838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088dba5-93d3-426d-bbee-34689fd85ba3 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:04.436188 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088dba5-93d3-426d-bbee-34689fd85ba3 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:04.436466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088dba5-93d3-426d-bbee-34689fd85ba3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:04.438847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b088dba5-93d3-426d-bbee-34689fd85ba3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:04.480534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:04.541563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088dba5-93d3-426d-bbee-34689fd85ba3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:04.543188 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088dba5-93d3-426d-bbee-34689fd85ba3 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:04.565779 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 697/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:06.674005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fdf48-6a4a-4224-9eee-3b299c1e9f04 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:06.674584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fdf48-6a4a-4224-9eee-3b299c1e9f04 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:06.674584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fdf48-6a4a-4224-9eee-3b299c1e9f04 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:06.676044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 710/1407] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:45:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:06.681683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163c1af-7de9-4a85-a5da-cc382195a865 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:06.682099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163c1af-7de9-4a85-a5da-cc382195a865 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:06.682370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163c1af-7de9-4a85-a5da-cc382195a865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:06.771116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-9163c1af-7de9-4a85-a5da-cc382195a865 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:06.772371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9163c1af-7de9-4a85-a5da-cc382195a865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:06.887570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 698/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:07.186676 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fdf48-6a4a-4224-9eee-3b299c1e9f04 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:07.188279 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2a5fdf48-6a4a-4224-9eee-3b299c1e9f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:07.286537 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:07.355540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fdf48-6a4a-4224-9eee-3b299c1e9f04 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:07.482973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef4e21-be7b-45f7-ae4b-5f4213dd8a12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:07.483876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef4e21-be7b-45f7-ae4b-5f4213dd8a12 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:07.484163 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef4e21-be7b-45f7-ae4b-5f4213dd8a12 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:07.484476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef4e21-be7b-45f7-ae4b-5f4213dd8a12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:07.486202 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-abef4e21-be7b-45f7-ae4b-5f4213dd8a12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:07.544930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:07.592101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef4e21-be7b-45f7-ae4b-5f4213dd8a12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:07.598187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef4e21-be7b-45f7-ae4b-5f4213dd8a12 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:07.617868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 711/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:16.885730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=93 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:17.615472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=24 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:23.375842 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28738d-3df1-4fcc-8d86-04ba8717a034 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:23.376441 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28738d-3df1-4fcc-8d86-04ba8717a034 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:23.376441 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28738d-3df1-4fcc-8d86-04ba8717a034 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:23.377850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 699/1410] 10.222.0.26 () {54 vars in 918 bytes} [Wed Aug 24 07:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:23.383320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db49aa1-d0a9-4d87-a0bf-b5ea93ce54e7 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:23.383768 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db49aa1-d0a9-4d87-a0bf-b5ea93ce54e7 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:23.383863 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db49aa1-d0a9-4d87-a0bf-b5ea93ce54e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:23.458516 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-5db49aa1-d0a9-4d87-a0bf-b5ea93ce54e7 None None] MFA Rules not processed for user `ea3816d5ae054741ac2c5a2eec3eb617`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:23.459959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5db49aa1-d0a9-4d87-a0bf-b5ea93ce54e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:23.564825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 712/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Aug 24 07:45:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:23.579845 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f097e4a-05e0-4a3c-9b4b-704f18be566e None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:23.580135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f097e4a-05e0-4a3c-9b4b-704f18be566e None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:23.580411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f097e4a-05e0-4a3c-9b4b-704f18be566e None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:23.582375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 700/1412] 10.222.0.26 () {54 vars in 918 bytes} [Wed Aug 24 07:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:23.587291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f06df-381b-430c-b4d1-686b85b1c0a4 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:23.587584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f06df-381b-430c-b4d1-686b85b1c0a4 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:23.587864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f06df-381b-430c-b4d1-686b85b1c0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:23.659819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:23.662442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-2d6f06df-381b-430c-b4d1-686b85b1c0a4 None None] MFA Rules not processed for user `bdf81f09006a42299c0807f77f16c748`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:23.663651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2d6f06df-381b-430c-b4d1-686b85b1c0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:23.778682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 713/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Aug 24 07:45:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:26.146154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed12cd-084b-431f-95b6-8fbba7815a47 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:26.146728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed12cd-084b-431f-95b6-8fbba7815a47 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:26.146728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed12cd-084b-431f-95b6-8fbba7815a47 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:26.148077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:26.153349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ca92f-fd3a-4c5d-bc3a-5508f198e9e4 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:26.153747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ca92f-fd3a-4c5d-bc3a-5508f198e9e4 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:26.153902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ca92f-fd3a-4c5d-bc3a-5508f198e9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:26.219218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-c93ca92f-fd3a-4c5d-bc3a-5508f198e9e4 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:26.220353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c93ca92f-fd3a-4c5d-bc3a-5508f198e9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:26.274893 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:26.315692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:26.607713 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed12cd-084b-431f-95b6-8fbba7815a47 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:26.609288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-33ed12cd-084b-431f-95b6-8fbba7815a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:26.755022 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed12cd-084b-431f-95b6-8fbba7815a47 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:26.826301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:26.868502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56ad7e-96cd-4b8c-9b9c-cf4411b153ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:26.869131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56ad7e-96cd-4b8c-9b9c-cf4411b153ed service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:26.869413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56ad7e-96cd-4b8c-9b9c-cf4411b153ed service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:26.869682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56ad7e-96cd-4b8c-9b9c-cf4411b153ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:26.871261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-df56ad7e-96cd-4b8c-9b9c-cf4411b153ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:26.959920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56ad7e-96cd-4b8c-9b9c-cf4411b153ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:26.961537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56ad7e-96cd-4b8c-9b9c-cf4411b153ed service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:26.978955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 702/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:27.866412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9971172-2b3d-4d03-b15d-c1728d90d096 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:27.866955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9971172-2b3d-4d03-b15d-c1728d90d096 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:27.866955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9971172-2b3d-4d03-b15d-c1728d90d096 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:27.868322 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:27.873223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464dbe9-3bb1-4daf-aca7-588554dcd479 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:27.873661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464dbe9-3bb1-4daf-aca7-588554dcd479 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:27.873756 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464dbe9-3bb1-4daf-aca7-588554dcd479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:27.949858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-0464dbe9-3bb1-4daf-aca7-588554dcd479 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:27.951058 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0464dbe9-3bb1-4daf-aca7-588554dcd479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:27.978164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:28.046919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:28.198631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9971172-2b3d-4d03-b15d-c1728d90d096 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:28.200134 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f9971172-2b3d-4d03-b15d-c1728d90d096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:28.346995 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9971172-2b3d-4d03-b15d-c1728d90d096 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:28.386449 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:28.458959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93887d3-774f-4a98-87e5-dd328773f135 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:28.459487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93887d3-774f-4a98-87e5-dd328773f135 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:28.460016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93887d3-774f-4a98-87e5-dd328773f135 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:28.460282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93887d3-774f-4a98-87e5-dd328773f135 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:28.461897 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e93887d3-774f-4a98-87e5-dd328773f135 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:28.550973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93887d3-774f-4a98-87e5-dd328773f135 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:28.552913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93887d3-774f-4a98-87e5-dd328773f135 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:28.571102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 716/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:29.462171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c412a68-c25a-4f1b-94a7-8462abc964a6 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:29.462729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c412a68-c25a-4f1b-94a7-8462abc964a6 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:29.462729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c412a68-c25a-4f1b-94a7-8462abc964a6 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:29.464141 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:29.469147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa7a3b-cb94-4566-b381-8ccc05d7eb30 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:29.469549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa7a3b-cb94-4566-b381-8ccc05d7eb30 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:29.469696 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa7a3b-cb94-4566-b381-8ccc05d7eb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:29.523187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:29.545003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-dcfa7a3b-cb94-4566-b381-8ccc05d7eb30 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:29.546223 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-dcfa7a3b-cb94-4566-b381-8ccc05d7eb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:29.642787 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:29.792616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c412a68-c25a-4f1b-94a7-8462abc964a6 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:29.794027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-8c412a68-c25a-4f1b-94a7-8462abc964a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:29.938588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c412a68-c25a-4f1b-94a7-8462abc964a6 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:29.951138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:30.055615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1523039-1bac-4602-93ab-71384d07e47a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:30.056228 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1523039-1bac-4602-93ab-71384d07e47a service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:30.056512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1523039-1bac-4602-93ab-71384d07e47a service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:30.056819 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1523039-1bac-4602-93ab-71384d07e47a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:30.058436 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b1523039-1bac-4602-93ab-71384d07e47a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:30.149128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1523039-1bac-4602-93ab-71384d07e47a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:30.150763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1523039-1bac-4602-93ab-71384d07e47a service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:30.168454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 705/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:31.027958 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d03fe-d13f-432a-a4ae-08a256ffbd3e None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:31.028500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d03fe-d13f-432a-a4ae-08a256ffbd3e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:31.028500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d03fe-d13f-432a-a4ae-08a256ffbd3e None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:31.029874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:31.035422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24218e3-a37d-4226-b141-138b8fca008e None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:31.035856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24218e3-a37d-4226-b141-138b8fca008e None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:31.036066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24218e3-a37d-4226-b141-138b8fca008e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:31.056599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:31.114751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-f24218e3-a37d-4226-b141-138b8fca008e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:31.115906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f24218e3-a37d-4226-b141-138b8fca008e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:31.210502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:31.363460 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d03fe-d13f-432a-a4ae-08a256ffbd3e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:31.365249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d97d03fe-d13f-432a-a4ae-08a256ffbd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:31.474666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:31.511551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d03fe-d13f-432a-a4ae-08a256ffbd3e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:31.622753 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed443cd4-3f8a-49a7-9a83-23cb426da259 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:31.623276 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed443cd4-3f8a-49a7-9a83-23cb426da259 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:31.623553 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed443cd4-3f8a-49a7-9a83-23cb426da259 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:31.623819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed443cd4-3f8a-49a7-9a83-23cb426da259 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:31.625431 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ed443cd4-3f8a-49a7-9a83-23cb426da259 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:31.700634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:31.714151 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed443cd4-3f8a-49a7-9a83-23cb426da259 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:31.715762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed443cd4-3f8a-49a7-9a83-23cb426da259 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:31.733900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 719/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:32.601144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d66cde-2299-43fa-8fea-d2a4f2803a6f None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:32.601717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d66cde-2299-43fa-8fea-d2a4f2803a6f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:32.601717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d66cde-2299-43fa-8fea-d2a4f2803a6f None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:32.603049 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:32.608095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2afda-324d-4368-805e-9b993598abfa None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:32.608593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2afda-324d-4368-805e-9b993598abfa None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:32.608736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2afda-324d-4368-805e-9b993598abfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:32.683912 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-53a2afda-324d-4368-805e-9b993598abfa None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:32.685133 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-53a2afda-324d-4368-805e-9b993598abfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:32.785477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:32.938881 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d66cde-2299-43fa-8fea-d2a4f2803a6f None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:32.940432 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c2d66cde-2299-43fa-8fea-d2a4f2803a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:33.025275 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:33.087281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d66cde-2299-43fa-8fea-d2a4f2803a6f None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:33.199027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea7387-64b3-46eb-af5b-4ec42bc631d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:33.199554 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea7387-64b3-46eb-af5b-4ec42bc631d3 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:33.199837 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea7387-64b3-46eb-af5b-4ec42bc631d3 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:33.200116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea7387-64b3-46eb-af5b-4ec42bc631d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:33.201763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-90ea7387-64b3-46eb-af5b-4ec42bc631d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:33.249669 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:33.289642 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea7387-64b3-46eb-af5b-4ec42bc631d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:33.291323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea7387-64b3-46eb-af5b-4ec42bc631d3 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:33.309775 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 708/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:34.233816 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a63a71-b152-4e3a-ae33-5d5dd7a063fa None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:34.234418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a63a71-b152-4e3a-ae33-5d5dd7a063fa None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:34.234418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a63a71-b152-4e3a-ae33-5d5dd7a063fa None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:34.235730 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:34.241064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12285bc5-ca51-4528-8c3d-414810bb707f None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:34.241483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12285bc5-ca51-4528-8c3d-414810bb707f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:34.241619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12285bc5-ca51-4528-8c3d-414810bb707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:34.318832 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-12285bc5-ca51-4528-8c3d-414810bb707f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:34.320037 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-12285bc5-ca51-4528-8c3d-414810bb707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:34.415424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:34.564510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a63a71-b152-4e3a-ae33-5d5dd7a063fa None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:34.566739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-45a63a71-b152-4e3a-ae33-5d5dd7a063fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:34.615886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:34.713142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a63a71-b152-4e3a-ae33-5d5dd7a063fa None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:34.824338 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287277e4-c3b1-451d-bd5e-5f2e8b16a0c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:34.824969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287277e4-c3b1-451d-bd5e-5f2e8b16a0c1 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:34.825255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287277e4-c3b1-451d-bd5e-5f2e8b16a0c1 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:34.825524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287277e4-c3b1-451d-bd5e-5f2e8b16a0c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:34.827103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-287277e4-c3b1-451d-bd5e-5f2e8b16a0c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:34.835138 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:34.919079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287277e4-c3b1-451d-bd5e-5f2e8b16a0c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:34.920735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287277e4-c3b1-451d-bd5e-5f2e8b16a0c1 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:34.939303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 722/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:35.849018 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20487a41-a058-4c2c-b869-76a8fa0c8cda None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:35.849577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20487a41-a058-4c2c-b869-76a8fa0c8cda None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:35.849577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20487a41-a058-4c2c-b869-76a8fa0c8cda None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:35.850974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:45:35.856013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224755-23f7-4ce6-a77c-5aa7f77ee3c2 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:35.856430 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224755-23f7-4ce6-a77c-5aa7f77ee3c2 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:35.856565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224755-23f7-4ce6-a77c-5aa7f77ee3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:35.931390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-08224755-23f7-4ce6-a77c-5aa7f77ee3c2 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:35.932593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-08224755-23f7-4ce6-a77c-5aa7f77ee3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:35.999344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:36.028003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:36.178982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20487a41-a058-4c2c-b869-76a8fa0c8cda None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:36.180543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-20487a41-a058-4c2c-b869-76a8fa0c8cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:36.198389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:36.327044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20487a41-a058-4c2c-b869-76a8fa0c8cda None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:36.413677 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:36.439604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6486a62-0403-4ea4-9cd3-c9f05094785d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:36.440137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6486a62-0403-4ea4-9cd3-c9f05094785d service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:36.440418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6486a62-0403-4ea4-9cd3-c9f05094785d service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:36.440726 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6486a62-0403-4ea4-9cd3-c9f05094785d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:36.443220 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b6486a62-0403-4ea4-9cd3-c9f05094785d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:36.533493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6486a62-0403-4ea4-9cd3-c9f05094785d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:36.535407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6486a62-0403-4ea4-9cd3-c9f05094785d service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:36.553178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 711/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:37.434572 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580d669-b1c6-4cb9-af95-78e52481a50d None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:37.435160 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580d669-b1c6-4cb9-af95-78e52481a50d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:37.435160 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580d669-b1c6-4cb9-af95-78e52481a50d None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:37.436470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:37.441994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696313d3-01ee-4d8b-92db-cf729cc576cf None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:37.442467 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696313d3-01ee-4d8b-92db-cf729cc576cf None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:37.442581 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696313d3-01ee-4d8b-92db-cf729cc576cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:37.518088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-696313d3-01ee-4d8b-92db-cf729cc576cf None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:37.519308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-696313d3-01ee-4d8b-92db-cf729cc576cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:37.676057 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:37.732438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:37.881801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580d669-b1c6-4cb9-af95-78e52481a50d None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:37.883313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5580d669-b1c6-4cb9-af95-78e52481a50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:38.029640 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580d669-b1c6-4cb9-af95-78e52481a50d None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:38.083865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:38.142126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39f59d-f7d1-45bf-8e50-6fa31d44db6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:38.142739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39f59d-f7d1-45bf-8e50-6fa31d44db6f service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:38.143018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39f59d-f7d1-45bf-8e50-6fa31d44db6f service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:38.143290 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39f59d-f7d1-45bf-8e50-6fa31d44db6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:38.145777 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0a39f59d-f7d1-45bf-8e50-6fa31d44db6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:38.234273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39f59d-f7d1-45bf-8e50-6fa31d44db6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:38.235887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39f59d-f7d1-45bf-8e50-6fa31d44db6f service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:38.254908 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 725/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:39.117682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd879e8-b158-4dc3-b9e1-2ea940fc348b None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:39.118239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd879e8-b158-4dc3-b9e1-2ea940fc348b None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:39.118239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd879e8-b158-4dc3-b9e1-2ea940fc348b None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:39.119470 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:39.124422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad5b00-4f87-49b3-a155-f1601d2ac859 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:39.124872 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad5b00-4f87-49b3-a155-f1601d2ac859 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:39.125008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad5b00-4f87-49b3-a155-f1601d2ac859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:39.201321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-92ad5b00-4f87-49b3-a155-f1601d2ac859 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:39.202574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-92ad5b00-4f87-49b3-a155-f1601d2ac859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:39.204603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:39.300588 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:39.451152 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd879e8-b158-4dc3-b9e1-2ea940fc348b None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:39.452589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-edd879e8-b158-4dc3-b9e1-2ea940fc348b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:39.598056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd879e8-b158-4dc3-b9e1-2ea940fc348b None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:39.624702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:39.711288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fd187-7a8b-4ae1-8dfb-80951a59b324 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:39.711813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fd187-7a8b-4ae1-8dfb-80951a59b324 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:39.712092 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fd187-7a8b-4ae1-8dfb-80951a59b324 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:39.712364 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fd187-7a8b-4ae1-8dfb-80951a59b324 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:39.714026 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5d6fd187-7a8b-4ae1-8dfb-80951a59b324 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:39.801904 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fd187-7a8b-4ae1-8dfb-80951a59b324 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:39.803489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fd187-7a8b-4ae1-8dfb-80951a59b324 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:39.821687 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 714/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:41.653285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f297a-9c9f-4a09-b377-efc2b463720e None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:41.653851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f297a-9c9f-4a09-b377-efc2b463720e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:41.653851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f297a-9c9f-4a09-b377-efc2b463720e None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:41.655192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:41.660681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2716a1c-4609-4db0-8173-e409c04a6c86 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:41.661120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2716a1c-4609-4db0-8173-e409c04a6c86 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:41.661232 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2716a1c-4609-4db0-8173-e409c04a6c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:41.701724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:41.727719 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-c2716a1c-4609-4db0-8173-e409c04a6c86 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:41.728944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c2716a1c-4609-4db0-8173-e409c04a6c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:41.823603 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:42.112169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f297a-9c9f-4a09-b377-efc2b463720e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:42.114433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-488f297a-9c9f-4a09-b377-efc2b463720e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:42.218612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f297a-9c9f-4a09-b377-efc2b463720e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:42.251037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:42.292367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78655fc0-825a-46a4-baa8-1fc8849ca677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:42.293006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78655fc0-825a-46a4-baa8-1fc8849ca677 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:42.293304 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78655fc0-825a-46a4-baa8-1fc8849ca677 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:42.293581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78655fc0-825a-46a4-baa8-1fc8849ca677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:42.295187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-78655fc0-825a-46a4-baa8-1fc8849ca677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:42.404256 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78655fc0-825a-46a4-baa8-1fc8849ca677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:42.405677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78655fc0-825a-46a4-baa8-1fc8849ca677 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:42.424176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 728/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:44.494640 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59e86a-ba37-45ca-b503-ad8c340e710f None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:44.495249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59e86a-ba37-45ca-b503-ad8c340e710f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:44.495249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59e86a-ba37-45ca-b503-ad8c340e710f None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:44.496557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:44.501666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3715e-2345-491d-8019-192e159a029f None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:44.502107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3715e-2345-491d-8019-192e159a029f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:44.502215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3715e-2345-491d-8019-192e159a029f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:44.568247 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-4cd3715e-2345-491d-8019-192e159a029f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:44.569484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4cd3715e-2345-491d-8019-192e159a029f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:44.582580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:44.666659 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:44.949404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59e86a-ba37-45ca-b503-ad8c340e710f None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:44.950933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0d59e86a-ba37-45ca-b503-ad8c340e710f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:45.053890 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59e86a-ba37-45ca-b503-ad8c340e710f None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:45.066316 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:45.127324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd58e4-111f-478e-9231-87776b095c4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:45.127862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd58e4-111f-478e-9231-87776b095c4c service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:45.128816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd58e4-111f-478e-9231-87776b095c4c service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:45.129134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd58e4-111f-478e-9231-87776b095c4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:45.130690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c4bd58e4-111f-478e-9231-87776b095c4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:45.239197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd58e4-111f-478e-9231-87776b095c4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:45.240717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd58e4-111f-478e-9231-87776b095c4c service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:45.258568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 717/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:47.577236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee317d4-cdf8-4de4-a090-165350e087d7 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:47.577872 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee317d4-cdf8-4de4-a090-165350e087d7 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:47.578155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee317d4-cdf8-4de4-a090-165350e087d7 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:47.579499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:47.585449 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d59a7-c349-45ba-97d9-ed472cd35fe6 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:47.585903 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d59a7-c349-45ba-97d9-ed472cd35fe6 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:47.586005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d59a7-c349-45ba-97d9-ed472cd35fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:47.646481 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:47.672230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e38d59a7-c349-45ba-97d9-ed472cd35fe6 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:47.675961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e38d59a7-c349-45ba-97d9-ed472cd35fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:47.798363 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:48.123906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee317d4-cdf8-4de4-a090-165350e087d7 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:48.125510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-eee317d4-cdf8-4de4-a090-165350e087d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:48.231283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee317d4-cdf8-4de4-a090-165350e087d7 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:48.272838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:48.307265 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c048bc7-cf61-4098-b8f7-6a23d78d545c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:48.308312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c048bc7-cf61-4098-b8f7-6a23d78d545c service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:48.308858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c048bc7-cf61-4098-b8f7-6a23d78d545c service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:48.309358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c048bc7-cf61-4098-b8f7-6a23d78d545c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:48.313255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8c048bc7-cf61-4098-b8f7-6a23d78d545c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:48.430543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c048bc7-cf61-4098-b8f7-6a23d78d545c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:48.431953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c048bc7-cf61-4098-b8f7-6a23d78d545c service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:48.450388 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 731/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:50.474588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57acbf-5166-48d9-a7ee-ec949190a7e9 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:50.475137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57acbf-5166-48d9-a7ee-ec949190a7e9 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:50.475137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57acbf-5166-48d9-a7ee-ec949190a7e9 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:50.476489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:50.481704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690240bd-5c92-419a-9268-6ae9f2a29515 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:50.482143 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690240bd-5c92-419a-9268-6ae9f2a29515 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:50.482258 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690240bd-5c92-419a-9268-6ae9f2a29515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:50.553079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-690240bd-5c92-419a-9268-6ae9f2a29515 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:50.554284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-690240bd-5c92-419a-9268-6ae9f2a29515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:50.575561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:50.651714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:50.938174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57acbf-5166-48d9-a7ee-ec949190a7e9 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:50.939695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6f57acbf-5166-48d9-a7ee-ec949190a7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:51.042456 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57acbf-5166-48d9-a7ee-ec949190a7e9 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:51.061877 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:51.114946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b74dda-ec52-4eb2-8d2c-91e722cbe5fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:51.115548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b74dda-ec52-4eb2-8d2c-91e722cbe5fb service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:51.115828 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b74dda-ec52-4eb2-8d2c-91e722cbe5fb service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:51.116100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b74dda-ec52-4eb2-8d2c-91e722cbe5fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:51.117737 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-89b74dda-ec52-4eb2-8d2c-91e722cbe5fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:51.224238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b74dda-ec52-4eb2-8d2c-91e722cbe5fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:51.225614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b74dda-ec52-4eb2-8d2c-91e722cbe5fb service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:51.243609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 720/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:53.215265 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba55bdf-f358-43d3-933f-68d304ba073c None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:53.215787 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba55bdf-f358-43d3-933f-68d304ba073c None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:53.215787 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba55bdf-f358-43d3-933f-68d304ba073c None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:53.217128 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:53.223118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e63c8-9781-45e6-bc5a-0c1fabe30715 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:53.223518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e63c8-9781-45e6-bc5a-0c1fabe30715 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:53.223659 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e63c8-9781-45e6-bc5a-0c1fabe30715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:53.291671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-ea0e63c8-9781-45e6-bc5a-0c1fabe30715 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:53.292797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:53.293666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ea0e63c8-9781-45e6-bc5a-0c1fabe30715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:53.388814 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:53.673299 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba55bdf-f358-43d3-933f-68d304ba073c None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:53.674879 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-aba55bdf-f358-43d3-933f-68d304ba073c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:53.778274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba55bdf-f358-43d3-933f-68d304ba073c None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:53.825421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:53.849815 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40f003-7d2c-4723-a7e6-2e3a3d60fa69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:53.850418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40f003-7d2c-4723-a7e6-2e3a3d60fa69 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:53.850695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40f003-7d2c-4723-a7e6-2e3a3d60fa69 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:53.850967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40f003-7d2c-4723-a7e6-2e3a3d60fa69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:53.852550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7d40f003-7d2c-4723-a7e6-2e3a3d60fa69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:53.961281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40f003-7d2c-4723-a7e6-2e3a3d60fa69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:53.962727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40f003-7d2c-4723-a7e6-2e3a3d60fa69 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:53.980734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 734/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:55.968431 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac55db9-94f1-4eaa-af7f-ff1512091e9d None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:55.969011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac55db9-94f1-4eaa-af7f-ff1512091e9d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:55.969108 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac55db9-94f1-4eaa-af7f-ff1512091e9d None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:55.977909 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:55.978092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e83fb-7166-4081-9ba2-070ee02ecc85 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:55.978092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e83fb-7166-4081-9ba2-070ee02ecc85 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:55.978092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e83fb-7166-4081-9ba2-070ee02ecc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:56.043413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-086e83fb-7166-4081-9ba2-070ee02ecc85 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:56.044778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-086e83fb-7166-4081-9ba2-070ee02ecc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:56.073542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:56.140484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:56.425099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac55db9-94f1-4eaa-af7f-ff1512091e9d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:56.426782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2ac55db9-94f1-4eaa-af7f-ff1512091e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:56.531543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac55db9-94f1-4eaa-af7f-ff1512091e9d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:56.560511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:56.606475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03f7b2-7815-429a-841e-dbb7bd2a7c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:56.607075 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03f7b2-7815-429a-841e-dbb7bd2a7c4e service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:56.607353 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03f7b2-7815-429a-841e-dbb7bd2a7c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:56.607626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03f7b2-7815-429a-841e-dbb7bd2a7c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:56.609258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2c03f7b2-7815-429a-841e-dbb7bd2a7c4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:56.718207 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03f7b2-7815-429a-841e-dbb7bd2a7c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:56.719586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03f7b2-7815-429a-841e-dbb7bd2a7c4e service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:56.737976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 723/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:45:58.686119 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ee119-0b87-4182-a99e-9a8e1bfa7c11 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:58.686722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ee119-0b87-4182-a99e-9a8e1bfa7c11 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:58.686722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ee119-0b87-4182-a99e-9a8e1bfa7c11 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:58.688065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:45:58.694278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d77553-6a19-4628-972b-ba0fcbdb5a7e None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:58.694698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d77553-6a19-4628-972b-ba0fcbdb5a7e None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:58.694859 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d77553-6a19-4628-972b-ba0fcbdb5a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:58.761136 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-57d77553-6a19-4628-972b-ba0fcbdb5a7e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:45:58.762369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-57d77553-6a19-4628-972b-ba0fcbdb5a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:58.771525 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:58.860374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:45:59.144891 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ee119-0b87-4182-a99e-9a8e1bfa7c11 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:45:59.146426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f61ee119-0b87-4182-a99e-9a8e1bfa7c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:59.249317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ee119-0b87-4182-a99e-9a8e1bfa7c11 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:45:59.304193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:45:59.322837 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28964715-0cc3-4d84-b6b4-c3213105ac5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:45:59.323464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28964715-0cc3-4d84-b6b4-c3213105ac5a service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:45:59.323752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28964715-0cc3-4d84-b6b4-c3213105ac5a service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:45:59.324025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28964715-0cc3-4d84-b6b4-c3213105ac5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:45:59.325650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-28964715-0cc3-4d84-b6b4-c3213105ac5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:45:59.433258 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28964715-0cc3-4d84-b6b4-c3213105ac5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:45:59.434631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28964715-0cc3-4d84-b6b4-c3213105ac5a service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:45:59.454464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 737/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:01.525841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a431b3f-f8d2-4cf7-9b7d-321b8d769c8a None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:01.526406 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a431b3f-f8d2-4cf7-9b7d-321b8d769c8a None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:01.526406 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a431b3f-f8d2-4cf7-9b7d-321b8d769c8a None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:01.527761 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:46:01.533297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c03f2-c147-4bba-b3ff-10b21db59f8f None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:01.533765 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c03f2-c147-4bba-b3ff-10b21db59f8f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:01.533996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c03f2-c147-4bba-b3ff-10b21db59f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:01.601045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-655c03f2-c147-4bba-b3ff-10b21db59f8f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:01.602328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-655c03f2-c147-4bba-b3ff-10b21db59f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:01.639773 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:01.697957 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:46:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:01.982479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a431b3f-f8d2-4cf7-9b7d-321b8d769c8a None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:01.984086 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5a431b3f-f8d2-4cf7-9b7d-321b8d769c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:02.087067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a431b3f-f8d2-4cf7-9b7d-321b8d769c8a None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:02.125158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:02.159532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3287fa-c429-42f4-bdc9-6a80f9735007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:02.160127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3287fa-c429-42f4-bdc9-6a80f9735007 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:02.160408 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3287fa-c429-42f4-bdc9-6a80f9735007 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:02.160708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3287fa-c429-42f4-bdc9-6a80f9735007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:02.162302 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fe3287fa-c429-42f4-bdc9-6a80f9735007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:02.271459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3287fa-c429-42f4-bdc9-6a80f9735007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:02.273055 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3287fa-c429-42f4-bdc9-6a80f9735007 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:02.291637 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 726/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:04.246097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4685eb-4793-43ef-b9c6-633377948cb5 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:04.246692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4685eb-4793-43ef-b9c6-633377948cb5 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:04.246692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4685eb-4793-43ef-b9c6-633377948cb5 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:04.247917 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:46:04.253563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f595dd-d1bf-4e16-94b2-cce51465a879 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:04.254010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f595dd-d1bf-4e16-94b2-cce51465a879 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:04.254160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f595dd-d1bf-4e16-94b2-cce51465a879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:04.319563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-46f595dd-d1bf-4e16-94b2-cce51465a879 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:04.320811 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-46f595dd-d1bf-4e16-94b2-cce51465a879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:04.339065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:04.417500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:04.701962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4685eb-4793-43ef-b9c6-633377948cb5 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:04.703507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-fe4685eb-4793-43ef-b9c6-633377948cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:04.806306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4685eb-4793-43ef-b9c6-633377948cb5 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:04.873762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:04.881508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9c564-0ca4-440a-8582-6f9eef5c2ef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:04.882118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9c564-0ca4-440a-8582-6f9eef5c2ef7 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:04.882404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9c564-0ca4-440a-8582-6f9eef5c2ef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:04.882685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9c564-0ca4-440a-8582-6f9eef5c2ef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:04.884299 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-69e9c564-0ca4-440a-8582-6f9eef5c2ef7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:04.993506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9c564-0ca4-440a-8582-6f9eef5c2ef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:04.994982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9c564-0ca4-440a-8582-6f9eef5c2ef7 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:05.013867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 740/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:06.943333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7d886-3af8-4abc-8379-74d86530d475 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:06.943879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7d886-3af8-4abc-8379-74d86530d475 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:06.943879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7d886-3af8-4abc-8379-74d86530d475 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:06.945290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:46:06.950587 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42decc02-154c-4d4b-947f-bb265ecc9ee5 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:06.951011 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42decc02-154c-4d4b-947f-bb265ecc9ee5 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:06.951141 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42decc02-154c-4d4b-947f-bb265ecc9ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:07.018791 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-42decc02-154c-4d4b-947f-bb265ecc9ee5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:07.020036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-42decc02-154c-4d4b-947f-bb265ecc9ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:07.068313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:07.117413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:46:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:07.407403 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7d886-3af8-4abc-8379-74d86530d475 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:07.409658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b0a7d886-3af8-4abc-8379-74d86530d475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:07.512973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7d886-3af8-4abc-8379-74d86530d475 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:07.560527 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:07.586321 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e6bb1-f11b-44f0-8c55-891fcad4cc80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:07.586924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e6bb1-f11b-44f0-8c55-891fcad4cc80 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:07.587203 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e6bb1-f11b-44f0-8c55-891fcad4cc80 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:07.587495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e6bb1-f11b-44f0-8c55-891fcad4cc80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:07.589103 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-218e6bb1-f11b-44f0-8c55-891fcad4cc80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:07.698049 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e6bb1-f11b-44f0-8c55-891fcad4cc80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:07.699462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e6bb1-f11b-44f0-8c55-891fcad4cc80 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:07.718098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 729/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:17.115586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=22 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:17.715763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=66 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:27.310995 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45801f-a572-4d73-b1fc-07ffe5f55f41 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:27.311524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45801f-a572-4d73-b1fc-07ffe5f55f41 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:27.311654 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45801f-a572-4d73-b1fc-07ffe5f55f41 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:27.312940 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 742/1471] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:46:27.318313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887f726-209c-4c47-bc95-1e80c67f42ec None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:27.318709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887f726-209c-4c47-bc95-1e80c67f42ec None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:27.318896 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887f726-209c-4c47-bc95-1e80c67f42ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:27.383085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-5887f726-209c-4c47-bc95-1e80c67f42ec None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:27.384279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5887f726-209c-4c47-bc95-1e80c67f42ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:27.478759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 730/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:27.779194 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c39496-6a08-47a1-97e0-852d785b1a83 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:27.779530 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c39496-6a08-47a1-97e0-852d785b1a83 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:27.779803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c39496-6a08-47a1-97e0-852d785b1a83 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:27.781178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 743/1473] 10.222.0.26 () {54 vars in 967 bytes} [Wed Aug 24 07:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:46:27.786551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d8e1f-8a58-41d9-a230-799c486ddedc None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:27.786840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d8e1f-8a58-41d9-a230-799c486ddedc None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:27.787119 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d8e1f-8a58-41d9-a230-799c486ddedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:27.862480 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-fa0d8e1f-8a58-41d9-a230-799c486ddedc None None] MFA Rules not processed for user `38de805ba5eb4aa58cf95d2082c2d216`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:27.863668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fa0d8e1f-8a58-41d9-a230-799c486ddedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:27.883776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:27.938260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 731/1474] 10.222.0.26 () {58 vars in 1064 bytes} [Wed Aug 24 07:46:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:27.946581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c39496-6a08-47a1-97e0-852d785b1a83 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:27.948095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c3c39496-6a08-47a1-97e0-852d785b1a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:28.055783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c39496-6a08-47a1-97e0-852d785b1a83 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:28.124074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae83b1-118e-4715-b39b-6a7b820891fb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38de805ba5eb4aa58cf95d2082c2d216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:28.124702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae83b1-118e-4715-b39b-6a7b820891fb service glance] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:28.124983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae83b1-118e-4715-b39b-6a7b820891fb service glance] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:28.125251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae83b1-118e-4715-b39b-6a7b820891fb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:28.126856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ddae83b1-118e-4715-b39b-6a7b820891fb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:28.180078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:28.233347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae83b1-118e-4715-b39b-6a7b820891fb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:28.234797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae83b1-118e-4715-b39b-6a7b820891fb service glance] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:28.254343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 744/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:30.257007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d407cc2-d323-48b9-9bfb-fc46454bdf92 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:30.257550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d407cc2-d323-48b9-9bfb-fc46454bdf92 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:30.257550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d407cc2-d323-48b9-9bfb-fc46454bdf92 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:30.258854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 732/1476] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:46:30.264199 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e04a10-215b-4bdf-a162-01b93bb0e785 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:30.264713 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e04a10-215b-4bdf-a162-01b93bb0e785 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:30.264855 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e04a10-215b-4bdf-a162-01b93bb0e785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:30.330066 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-21e04a10-215b-4bdf-a162-01b93bb0e785 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:30.331343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-21e04a10-215b-4bdf-a162-01b93bb0e785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:30.545393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 745/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:30.819801 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d407cc2-d323-48b9-9bfb-fc46454bdf92 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:30.821386 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9d407cc2-d323-48b9-9bfb-fc46454bdf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:30.946266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d407cc2-d323-48b9-9bfb-fc46454bdf92 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:30.994538 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:31.038747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f3640-4a4c-4ad6-aba4-7ae23f8a09ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:31.039377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3640-4a4c-4ad6-aba4-7ae23f8a09ba admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:31.039658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3640-4a4c-4ad6-aba4-7ae23f8a09ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:31.039932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3640-4a4c-4ad6-aba4-7ae23f8a09ba admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:31.040567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f3640-4a4c-4ad6-aba4-7ae23f8a09ba admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:31.042316 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: WARNING py.warnings [None req-fb1f3640-4a4c-4ad6-aba4-7ae23f8a09ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:46:31.042316 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: warnings.warn(msg) Aug 24 07:46:31.042316 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: Aug 24 07:46:31.043447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f3640-4a4c-4ad6-aba4-7ae23f8a09ba admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:31.062243 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 733/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 24 07:46:30 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:46:33.000366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53716b79-9ea0-4f86-962b-d5f68eb73532 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:33.000954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53716b79-9ea0-4f86-962b-d5f68eb73532 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:33.000954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53716b79-9ea0-4f86-962b-d5f68eb73532 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:33.002998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 746/1479] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:46:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:46:33.008413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580740a3-5fb9-4fb5-b25b-2da3940e0567 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:33.008860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580740a3-5fb9-4fb5-b25b-2da3940e0567 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:33.009077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580740a3-5fb9-4fb5-b25b-2da3940e0567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:33.075366 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-580740a3-5fb9-4fb5-b25b-2da3940e0567 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:33.076882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-580740a3-5fb9-4fb5-b25b-2da3940e0567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:33.171491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 734/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:33.466395 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53716b79-9ea0-4f86-962b-d5f68eb73532 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:33.467910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-53716b79-9ea0-4f86-962b-d5f68eb73532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:33.478296 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:33.615073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53716b79-9ea0-4f86-962b-d5f68eb73532 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:33.695324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:33.728184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c719c-c2f8-4a40-b398-a96311d2dfb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:33.728996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c719c-c2f8-4a40-b398-a96311d2dfb8 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:33.729470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c719c-c2f8-4a40-b398-a96311d2dfb8 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:33.730058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c719c-c2f8-4a40-b398-a96311d2dfb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:33.731649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-167c719c-c2f8-4a40-b398-a96311d2dfb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:33.817594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c719c-c2f8-4a40-b398-a96311d2dfb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:33.819181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c719c-c2f8-4a40-b398-a96311d2dfb8 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:33.837404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 747/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:35.960368 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcf991-8ee0-4f90-af83-d5b92dac754c None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:35.961026 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcf991-8ee0-4f90-af83-d5b92dac754c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:35.961026 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcf991-8ee0-4f90-af83-d5b92dac754c None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:35.962354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 735/1482] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:46:35.967620 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41ee63-92d6-4c7f-a096-79eef55ea2b3 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:35.968091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41ee63-92d6-4c7f-a096-79eef55ea2b3 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:35.968200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41ee63-92d6-4c7f-a096-79eef55ea2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:36.034509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-ba41ee63-92d6-4c7f-a096-79eef55ea2b3 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:36.035753 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ba41ee63-92d6-4c7f-a096-79eef55ea2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:36.082523 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:36.132945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 748/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:46:36.425341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcf991-8ee0-4f90-af83-d5b92dac754c None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:36.426912 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c9bcf991-8ee0-4f90-af83-d5b92dac754c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:36.450428 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:36.530952 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcf991-8ee0-4f90-af83-d5b92dac754c None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:36.604743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167b2f0-ff64-426d-9652-6db3346ef4d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:36.605341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167b2f0-ff64-426d-9652-6db3346ef4d8 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:36.605616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167b2f0-ff64-426d-9652-6db3346ef4d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:36.605886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167b2f0-ff64-426d-9652-6db3346ef4d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:36.607473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4167b2f0-ff64-426d-9652-6db3346ef4d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:36.681812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:36.716216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167b2f0-ff64-426d-9652-6db3346ef4d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:36.717667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167b2f0-ff64-426d-9652-6db3346ef4d8 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:36.735169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 736/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:38.718623 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b7d10-06ca-4535-b71f-10d4fd247e94 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:38.719164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b7d10-06ca-4535-b71f-10d4fd247e94 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:38.719164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b7d10-06ca-4535-b71f-10d4fd247e94 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:38.720466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 749/1485] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:46:38.725826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bd9cc-8da2-4817-92c4-c4ef2db21ff3 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:38.726259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bd9cc-8da2-4817-92c4-c4ef2db21ff3 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:38.726369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bd9cc-8da2-4817-92c4-c4ef2db21ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:38.791015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-a33bd9cc-8da2-4817-92c4-c4ef2db21ff3 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:38.792209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-a33bd9cc-8da2-4817-92c4-c4ef2db21ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:38.886857 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 737/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:39.173390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b7d10-06ca-4535-b71f-10d4fd247e94 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:39.174935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-350b7d10-06ca-4535-b71f-10d4fd247e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:39.276014 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b7d10-06ca-4535-b71f-10d4fd247e94 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:39.346567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:39.348351 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7f62a-85f5-4735-8b95-9d8a2023391f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:39.348972 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7f62a-85f5-4735-8b95-9d8a2023391f service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:39.349249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7f62a-85f5-4735-8b95-9d8a2023391f service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:39.349517 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7f62a-85f5-4735-8b95-9d8a2023391f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:39.351095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c6c7f62a-85f5-4735-8b95-9d8a2023391f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:39.457921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7f62a-85f5-4735-8b95-9d8a2023391f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:39.459559 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7f62a-85f5-4735-8b95-9d8a2023391f service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:39.476827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 750/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:41.492800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2a143-0edd-4658-9dfc-707bc6748cd0 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:41.493374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2a143-0edd-4658-9dfc-707bc6748cd0 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:41.493374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2a143-0edd-4658-9dfc-707bc6748cd0 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:41.494722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 738/1488] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:46:41.499992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1711a-5ac4-485b-8ae0-d37c7b0d52f5 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:41.500434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1711a-5ac4-485b-8ae0-d37c7b0d52f5 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:41.500546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1711a-5ac4-485b-8ae0-d37c7b0d52f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:41.567367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-59b1711a-5ac4-485b-8ae0-d37c7b0d52f5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:41.568570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-59b1711a-5ac4-485b-8ae0-d37c7b0d52f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:41.620185 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:41.663273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 751/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:41.953809 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2a143-0edd-4658-9dfc-707bc6748cd0 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:41.955517 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-17f2a143-0edd-4658-9dfc-707bc6748cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:41.991811 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:42.060079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2a143-0edd-4658-9dfc-707bc6748cd0 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:42.135464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55377a89-2f9e-4d67-a59f-104ae29cce65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:42.136088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55377a89-2f9e-4d67-a59f-104ae29cce65 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:42.136370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55377a89-2f9e-4d67-a59f-104ae29cce65 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:42.136658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55377a89-2f9e-4d67-a59f-104ae29cce65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:42.138287 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-55377a89-2f9e-4d67-a59f-104ae29cce65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:42.219375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:42.250282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55377a89-2f9e-4d67-a59f-104ae29cce65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:42.252238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55377a89-2f9e-4d67-a59f-104ae29cce65 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:42.270163 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 739/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:44.557943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4114f2d-619c-4746-8136-6f17518ad53e None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:44.558416 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4114f2d-619c-4746-8136-6f17518ad53e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:44.558518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4114f2d-619c-4746-8136-6f17518ad53e None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:44.559850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 752/1491] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 24 07:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:46:44.565833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f3b48-338f-40cf-8083-f3da5e821b05 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:44.566278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f3b48-338f-40cf-8083-f3da5e821b05 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:44.566385 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f3b48-338f-40cf-8083-f3da5e821b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:44.632736 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-c30f3b48-338f-40cf-8083-f3da5e821b05 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:46:44.633948 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c30f3b48-338f-40cf-8083-f3da5e821b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:44.729476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 740/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 24 07:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:46:45.016235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4114f2d-619c-4746-8136-6f17518ad53e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:46:45.018419 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f4114f2d-619c-4746-8136-6f17518ad53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:45.121804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4114f2d-619c-4746-8136-6f17518ad53e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:46:45.194227 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e1d0c-66a9-4f07-a2cf-d51dc5fdb11e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:46:45.194755 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e1d0c-66a9-4f07-a2cf-d51dc5fdb11e service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:46:45.195036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e1d0c-66a9-4f07-a2cf-d51dc5fdb11e service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:46:45.195305 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e1d0c-66a9-4f07-a2cf-d51dc5fdb11e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:46:45.196911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-543e1d0c-66a9-4f07-a2cf-d51dc5fdb11e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:46:45.206414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:45.303984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e1d0c-66a9-4f07-a2cf-d51dc5fdb11e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:46:45.305453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e1d0c-66a9-4f07-a2cf-d51dc5fdb11e service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:46:45.323603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 753/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:46:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:46:54.727722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=82 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:46:55.321366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=50 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:22.581165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bd785-a4dc-41c3-82b3-50b28dc452c6 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:22.583969 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bd785-a4dc-41c3-82b3-50b28dc452c6 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:22.583969 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bd785-a4dc-41c3-82b3-50b28dc452c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:22.658478 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-c80bd785-a4dc-41c3-82b3-50b28dc452c6 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:22.660283 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c80bd785-a4dc-41c3-82b3-50b28dc452c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:22.755431 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 741/1494] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:22.763584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d5412-914a-4b8d-980f-81dffce5c7f3 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:22.764074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d5412-914a-4b8d-980f-81dffce5c7f3 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:22.764182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d5412-914a-4b8d-980f-81dffce5c7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:22.841155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-e99d5412-914a-4b8d-980f-81dffce5c7f3 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:22.842380 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e99d5412-914a-4b8d-980f-81dffce5c7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:22.937301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 754/1495] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:22.942129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bd785-a4dc-41c3-82b3-50b28dc452c6 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:22.943371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c80bd785-a4dc-41c3-82b3-50b28dc452c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:22.954690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bd785-a4dc-41c3-82b3-50b28dc452c6 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:23.016332 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:23.049712 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224bbb61-16de-47bc-acd4-8922e7e15d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:23.050310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224bbb61-16de-47bc-acd4-8922e7e15d21 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:23.050593 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224bbb61-16de-47bc-acd4-8922e7e15d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:23.050867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224bbb61-16de-47bc-acd4-8922e7e15d21 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:23.051501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224bbb61-16de-47bc-acd4-8922e7e15d21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:48:23.051802 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224bbb61-16de-47bc-acd4-8922e7e15d21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:23.052841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: WARNING py.warnings [None req-224bbb61-16de-47bc-acd4-8922e7e15d21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:48:23.052841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: warnings.warn(msg) Aug 24 07:48:23.052841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: Aug 24 07:48:23.053726 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224bbb61-16de-47bc-acd4-8922e7e15d21 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:23.064564 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 742/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:48:23.073590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d5412-914a-4b8d-980f-81dffce5c7f3 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:23.083796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d5412-914a-4b8d-980f-81dffce5c7f3 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:23.144383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:23.177686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875329e2-d4d3-47e7-998c-1965a0c3ac9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:23.178480 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875329e2-d4d3-47e7-998c-1965a0c3ac9c admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:23.178957 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875329e2-d4d3-47e7-998c-1965a0c3ac9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:23.179549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875329e2-d4d3-47e7-998c-1965a0c3ac9c admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:23.180179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875329e2-d4d3-47e7-998c-1965a0c3ac9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:23.181927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: WARNING py.warnings [None req-875329e2-d4d3-47e7-998c-1965a0c3ac9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:48:23.181927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: warnings.warn(msg) Aug 24 07:48:23.181927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: Aug 24 07:48:23.183106 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875329e2-d4d3-47e7-998c-1965a0c3ac9c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:23.300609 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 755/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:48:23 2022] POST /identity/v3/projects => generated 379 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:48:23.304845 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224bbb61-16de-47bc-acd4-8922e7e15d21 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:23.306384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-224bbb61-16de-47bc-acd4-8922e7e15d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:23.341569 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224bbb61-16de-47bc-acd4-8922e7e15d21 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:23.436984 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caec397-fe05-4dd6-96be-23476a07b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:23.437484 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caec397-fe05-4dd6-96be-23476a07b7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:23.437764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caec397-fe05-4dd6-96be-23476a07b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:23.438037 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caec397-fe05-4dd6-96be-23476a07b7d9 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:23.438660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caec397-fe05-4dd6-96be-23476a07b7d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:23.439676 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: WARNING py.warnings [None req-4caec397-fe05-4dd6-96be-23476a07b7d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:48:23.439676 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: warnings.warn(msg) Aug 24 07:48:23.439676 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: Aug 24 07:48:23.441659 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caec397-fe05-4dd6-96be-23476a07b7d9 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:23.574529 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 743/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:48:23 2022] POST /identity/v3/users => generated 290 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:48:23.578652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875329e2-d4d3-47e7-998c-1965a0c3ac9c admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:23.580156 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-875329e2-d4d3-47e7-998c-1965a0c3ac9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:23.641900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875329e2-d4d3-47e7-998c-1965a0c3ac9c admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:23.724913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:23.734751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9ca1a-66cd-4561-b805-6a95025f3aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:23.738513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9ca1a-66cd-4561-b805-6a95025f3aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:23.738513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9ca1a-66cd-4561-b805-6a95025f3aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:23.738513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9ca1a-66cd-4561-b805-6a95025f3aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:23.738513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9ca1a-66cd-4561-b805-6a95025f3aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:23.738513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: WARNING py.warnings [None req-3ed9ca1a-66cd-4561-b805-6a95025f3aa2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:48:23.738513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: warnings.warn(msg) Aug 24 07:48:23.738513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: Aug 24 07:48:23.739141 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9ca1a-66cd-4561-b805-6a95025f3aa2 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:23.751235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 756/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:48:23.755992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caec397-fe05-4dd6-96be-23476a07b7d9 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:23.757603 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4caec397-fe05-4dd6-96be-23476a07b7d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:23.769379 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caec397-fe05-4dd6-96be-23476a07b7d9 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:23.795358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:23.865327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad721c-3645-4881-a395-43a46b069483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:23.865929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad721c-3645-4881-a395-43a46b069483 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:23.866213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad721c-3645-4881-a395-43a46b069483 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:23.866545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad721c-3645-4881-a395-43a46b069483 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:23.867189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad721c-3645-4881-a395-43a46b069483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:23.868208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: WARNING py.warnings [None req-e7ad721c-3645-4881-a395-43a46b069483 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:48:23.868208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: warnings.warn(msg) Aug 24 07:48:23.868208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: Aug 24 07:48:23.869156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad721c-3645-4881-a395-43a46b069483 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:23.881482 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 744/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:48:23.885736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9ca1a-66cd-4561-b805-6a95025f3aa2 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:23.887313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3ed9ca1a-66cd-4561-b805-6a95025f3aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:23.921278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9ca1a-66cd-4561-b805-6a95025f3aa2 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:24.016374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea457a-b072-452f-97cb-0973d5db0319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:24.016987 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea457a-b072-452f-97cb-0973d5db0319 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:24.017265 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea457a-b072-452f-97cb-0973d5db0319 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:24.017539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea457a-b072-452f-97cb-0973d5db0319 admin admin] PATH_INFO: `/v3/projects/6752bb885e824d1782855dc15850cb3a/users/eea8efdf0a3c4ffe85ed37292185c08d/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:24.039627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea457a-b072-452f-97cb-0973d5db0319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6752bb885e824d1782855dc15850cb3a, user_id=eea8efdf0a3c4ffe85ed37292185c08d, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:24.040705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: WARNING py.warnings [None req-72ea457a-b072-452f-97cb-0973d5db0319 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:48:24.040705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: warnings.warn(msg) Aug 24 07:48:24.040705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: Aug 24 07:48:24.042652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea457a-b072-452f-97cb-0973d5db0319 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:24.167332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 757/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:48:23 2022] PUT /identity/v3/projects/6752bb885e824d1782855dc15850cb3a/users/eea8efdf0a3c4ffe85ed37292185c08d/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:48:24.186413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add226b-a2e8-4ad2-a7be-4554feed5450 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:24.186702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add226b-a2e8-4ad2-a7be-4554feed5450 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:24.186977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add226b-a2e8-4ad2-a7be-4554feed5450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:24.243361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-7add226b-a2e8-4ad2-a7be-4554feed5450 None None] MFA Rules not processed for user `eea8efdf0a3c4ffe85ed37292185c08d`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:24.244552 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-7add226b-a2e8-4ad2-a7be-4554feed5450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:24.321015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 745/1502] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:48:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:48:24.334940 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cb756-8b99-463d-b2d1-4a569bc5af93 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:24.335263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cb756-8b99-463d-b2d1-4a569bc5af93 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:24.335539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cb756-8b99-463d-b2d1-4a569bc5af93 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:24.336918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 758/1503] 10.222.0.26 () {54 vars in 964 bytes} [Wed Aug 24 07:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:24.341670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd177cae-5458-4e02-b3c7-f3b13a70e7c4 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:24.341965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd177cae-5458-4e02-b3c7-f3b13a70e7c4 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:24.342243 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd177cae-5458-4e02-b3c7-f3b13a70e7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:24.373372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:24.418045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-cd177cae-5458-4e02-b3c7-f3b13a70e7c4 None None] MFA Rules not processed for user `43e57131a5be4af785166bc03a3337a2`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:24.419249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-cd177cae-5458-4e02-b3c7-f3b13a70e7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:24.493585 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 746/1504] 10.222.0.26 () {58 vars in 1061 bytes} [Wed Aug 24 07:48:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:48:24.501510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cb756-8b99-463d-b2d1-4a569bc5af93 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:24.503590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-668cb756-8b99-463d-b2d1-4a569bc5af93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:24.553168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:24.606697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cb756-8b99-463d-b2d1-4a569bc5af93 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:24.679867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269aaca0-f22d-4ecb-bd9c-fd6af30cb430 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:24.680458 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269aaca0-f22d-4ecb-bd9c-fd6af30cb430 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:24.680763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269aaca0-f22d-4ecb-bd9c-fd6af30cb430 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:24.681032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269aaca0-f22d-4ecb-bd9c-fd6af30cb430 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:24.682611 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-269aaca0-f22d-4ecb-bd9c-fd6af30cb430 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:24.778618 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269aaca0-f22d-4ecb-bd9c-fd6af30cb430 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:24.780038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269aaca0-f22d-4ecb-bd9c-fd6af30cb430 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:24.783457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:24.797670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 759/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:24.843595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8fe5b-0d03-4920-8054-8b47379502b5 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:24.844013 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8fe5b-0d03-4920-8054-8b47379502b5 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:24.844158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8fe5b-0d03-4920-8054-8b47379502b5 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:24.845448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 747/1506] 10.222.0.26 () {52 vars in 830 bytes} [Wed Aug 24 07:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:24.849041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269aaca0-f22d-4ecb-bd9c-fd6af30cb430 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:24.850281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-269aaca0-f22d-4ecb-bd9c-fd6af30cb430 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:24.946908 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269aaca0-f22d-4ecb-bd9c-fd6af30cb430 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:25.037074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:25.037692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:25.037968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:25.038233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] PATH_INFO: `/v3/users/eea8efdf0a3c4ffe85ed37292185c08d` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:25.050493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea8efdf0a3c4ffe85ed37292185c08d)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:25.051511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: WARNING py.warnings [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:48:25.051511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: warnings.warn(msg) Aug 24 07:48:25.051511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: Aug 24 07:48:25.052743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:25.062197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:25.250239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea8efdf0a3c4ffe85ed37292185c08d'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:25.266756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea8efdf0a3c4ffe85ed37292185c08d'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:25.344544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea8efdf0a3c4ffe85ed37292185c08d'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:25.345290 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea8efdf0a3c4ffe85ed37292185c08d'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:25.370935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea8efdf0a3c4ffe85ed37292185c08d'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:25.378972 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 760/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:48:24 2022] DELETE /identity/v3/users/eea8efdf0a3c4ffe85ed37292185c08d => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:25.383453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e75df6-4517-406d-b1b0-e7e0045a6011 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:25.383744 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e75df6-4517-406d-b1b0-e7e0045a6011 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:25.387962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e75df6-4517-406d-b1b0-e7e0045a6011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:25.443491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-41e75df6-4517-406d-b1b0-e7e0045a6011 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:25.444721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-41e75df6-4517-406d-b1b0-e7e0045a6011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:25.521718 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:25.536689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 748/1508] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:25.545111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:25.546341 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:25.646252 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165b9fe-a1ea-4b4e-a105-af8d2a564c15 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:25.701022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:25.719342 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3729e-00a3-469f-ad13-2fa13044180c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:25.719945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3729e-00a3-469f-ad13-2fa13044180c service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:25.720221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3729e-00a3-469f-ad13-2fa13044180c service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:25.720488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3729e-00a3-469f-ad13-2fa13044180c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:25.722477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8dc3729e-00a3-469f-ad13-2fa13044180c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:25.835572 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3729e-00a3-469f-ad13-2fa13044180c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:25.837663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3729e-00a3-469f-ad13-2fa13044180c service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:25.856933 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 761/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:26.341189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e75df6-4517-406d-b1b0-e7e0045a6011 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:26.342552 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-41e75df6-4517-406d-b1b0-e7e0045a6011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:26.354954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e75df6-4517-406d-b1b0-e7e0045a6011 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:26.454074 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:26.454696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:26.454975 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:26.455756 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac admin admin] PATH_INFO: `/v3/projects/6752bb885e824d1782855dc15850cb3a` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:26.466256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=6752bb885e824d1782855dc15850cb3a)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:26.467288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: WARNING py.warnings [None req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:48:26.467288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: warnings.warn(msg) Aug 24 07:48:26.467288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: Aug 24 07:48:26.468487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:26.576767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6752bb885e824d1782855dc15850cb3a', 'request_id': 'req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:26.590466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6752bb885e824d1782855dc15850cb3a', 'request_id': 'req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:26.647192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6752bb885e824d1782855dc15850cb3a', 'request_id': 'req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:26.670612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac admin admin] The token cache is being invalidate because project 6752bb885e824d1782855dc15850cb3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:48:26.670905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:26.671337 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-90ffd8f9-d4d3-40a3-bf2c-0f34ba98c3ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:48:26.672871 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 749/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:48:26 2022] DELETE /identity/v3/projects/6752bb885e824d1782855dc15850cb3a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:48:35.854707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=62 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:36.668703 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:52.848364 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0ddaa-28ca-4119-a8a5-9879b2e9f53a None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:52.848985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0ddaa-28ca-4119-a8a5-9879b2e9f53a None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:52.848985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0ddaa-28ca-4119-a8a5-9879b2e9f53a None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:52.850374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:52.855124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f5f34-3042-4c0a-a9fb-8f983aff1b7a None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:52.855563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f5f34-3042-4c0a-a9fb-8f983aff1b7a None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:52.855710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f5f34-3042-4c0a-a9fb-8f983aff1b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:52.935360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-646f5f34-3042-4c0a-a9fb-8f983aff1b7a None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:52.936570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-646f5f34-3042-4c0a-a9fb-8f983aff1b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:53.040053 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:53.192313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0ddaa-28ca-4119-a8a5-9879b2e9f53a None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:53.194515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c2a0ddaa-28ca-4119-a8a5-9879b2e9f53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:53.311314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0ddaa-28ca-4119-a8a5-9879b2e9f53a None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:53.317823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420d0e9-23c7-47b0-970d-408bd0554f46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:53.318455 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420d0e9-23c7-47b0-970d-408bd0554f46 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:53.318740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420d0e9-23c7-47b0-970d-408bd0554f46 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:53.319016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420d0e9-23c7-47b0-970d-408bd0554f46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:53.326961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420d0e9-23c7-47b0-970d-408bd0554f46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:53.329161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420d0e9-23c7-47b0-970d-408bd0554f46 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:53.338818 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 763/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:54.184555 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d20975-7f3e-43de-ba1c-86aa96be7ac4 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:54.185120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d20975-7f3e-43de-ba1c-86aa96be7ac4 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:54.185120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d20975-7f3e-43de-ba1c-86aa96be7ac4 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:54.186477 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:54.191581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16373c86-291c-4bc9-8e25-e490affc7034 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:54.192021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16373c86-291c-4bc9-8e25-e490affc7034 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:54.192129 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16373c86-291c-4bc9-8e25-e490affc7034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:54.212257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-16373c86-291c-4bc9-8e25-e490affc7034 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:54.213499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-16373c86-291c-4bc9-8e25-e490affc7034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:54.222078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:54.375400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d20975-7f3e-43de-ba1c-86aa96be7ac4 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:54.389749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d20975-7f3e-43de-ba1c-86aa96be7ac4 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:54.395829 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1564a8-7812-4f73-adba-587fb6536b64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:54.396422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1564a8-7812-4f73-adba-587fb6536b64 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:54.396729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1564a8-7812-4f73-adba-587fb6536b64 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:54.397006 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1564a8-7812-4f73-adba-587fb6536b64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:54.405122 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1564a8-7812-4f73-adba-587fb6536b64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:54.407413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1564a8-7812-4f73-adba-587fb6536b64 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:54.416846 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 752/1516] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:57.130544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79a55c-2d1e-48b3-b8a1-acb96a726bf4 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:57.131097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79a55c-2d1e-48b3-b8a1-acb96a726bf4 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:57.131097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79a55c-2d1e-48b3-b8a1-acb96a726bf4 None None] PATH_INFO: `/v3` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:57.132383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 765/1517] 10.222.0.15 () {54 vars in 925 bytes} [Wed Aug 24 07:49:57 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:49:57.138704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534e75b-0705-4763-bb74-62d85220dde9 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:57.139151 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534e75b-0705-4763-bb74-62d85220dde9 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:57.139255 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534e75b-0705-4763-bb74-62d85220dde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:57.198458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-0534e75b-0705-4763-bb74-62d85220dde9 None None] MFA Rules not processed for user `ea3816d5ae054741ac2c5a2eec3eb617`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:57.199669 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0534e75b-0705-4763-bb74-62d85220dde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:57.217099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:57.271726 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 753/1518] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Aug 24 07:49:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:01.289526 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab4189-2720-4db9-a602-52673e53bdb1 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:01.290058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab4189-2720-4db9-a602-52673e53bdb1 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:01.290058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab4189-2720-4db9-a602-52673e53bdb1 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:01.291320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 766/1519] 10.222.0.26 () {54 vars in 961 bytes} [Wed Aug 24 07:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:50:01.296216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6c776-e004-4ed8-aa9c-daedd37258ac None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:01.297202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6c776-e004-4ed8-aa9c-daedd37258ac None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:01.297483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6c776-e004-4ed8-aa9c-daedd37258ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:01.318028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-61b6c776-e004-4ed8-aa9c-daedd37258ac None None] MFA Rules not processed for user `ea3816d5ae054741ac2c5a2eec3eb617`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:01.319215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-61b6c776-e004-4ed8-aa9c-daedd37258ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:01.328247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 754/1520] 10.222.0.26 () {58 vars in 1058 bytes} [Wed Aug 24 07:50:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:01.335775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab4189-2720-4db9-a602-52673e53bdb1 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:01.349512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab4189-2720-4db9-a602-52673e53bdb1 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:01.354432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f65bf-887b-478f-8612-885f7dd96054 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:01.355022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f65bf-887b-478f-8612-885f7dd96054 service placement] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:01.355306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f65bf-887b-478f-8612-885f7dd96054 service placement] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:01.355583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f65bf-887b-478f-8612-885f7dd96054 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:01.364202 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f65bf-887b-478f-8612-885f7dd96054 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:01.365793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f65bf-887b-478f-8612-885f7dd96054 service placement] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:01.375219 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 767/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:05.654757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aec7c6-3630-4997-814b-0ef49e3caa78 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:05.655320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aec7c6-3630-4997-814b-0ef49e3caa78 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:05.655320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aec7c6-3630-4997-814b-0ef49e3caa78 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:05.657212 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:50:05.662053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d2c0f-e0ed-4cd4-9c6f-35d425dbe01f None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:05.662510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d2c0f-e0ed-4cd4-9c6f-35d425dbe01f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:05.662616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d2c0f-e0ed-4cd4-9c6f-35d425dbe01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:05.681939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-291d2c0f-e0ed-4cd4-9c6f-35d425dbe01f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:05.683150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-291d2c0f-e0ed-4cd4-9c6f-35d425dbe01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:05.691284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:05.845464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aec7c6-3630-4997-814b-0ef49e3caa78 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:05.860573 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aec7c6-3630-4997-814b-0ef49e3caa78 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:05.866404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8992d77-3a5f-4e07-88aa-430c24f379ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:05.867001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8992d77-3a5f-4e07-88aa-430c24f379ac service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:05.867277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8992d77-3a5f-4e07-88aa-430c24f379ac service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:05.867577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8992d77-3a5f-4e07-88aa-430c24f379ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:05.875780 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8992d77-3a5f-4e07-88aa-430c24f379ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:05.877400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8992d77-3a5f-4e07-88aa-430c24f379ac service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:05.886720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 756/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:06.713058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe985a-8013-4720-9578-1e3454c0d662 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:06.713670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe985a-8013-4720-9578-1e3454c0d662 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:06.713670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe985a-8013-4720-9578-1e3454c0d662 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:06.714964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:50:06.719955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fde65c-6d36-4e4c-a741-933d95988351 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:06.720397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fde65c-6d36-4e4c-a741-933d95988351 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:06.720579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fde65c-6d36-4e4c-a741-933d95988351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:06.739537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-57fde65c-6d36-4e4c-a741-933d95988351 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:06.740752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-57fde65c-6d36-4e4c-a741-933d95988351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:06.748442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:06.902716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe985a-8013-4720-9578-1e3454c0d662 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:06.916998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe985a-8013-4720-9578-1e3454c0d662 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:06.922948 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502bd3c-f6c9-4551-b2d1-8cfcc9e32c31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:06.923539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502bd3c-f6c9-4551-b2d1-8cfcc9e32c31 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:06.923837 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502bd3c-f6c9-4551-b2d1-8cfcc9e32c31 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:06.924096 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502bd3c-f6c9-4551-b2d1-8cfcc9e32c31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:06.932027 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502bd3c-f6c9-4551-b2d1-8cfcc9e32c31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:06.933643 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502bd3c-f6c9-4551-b2d1-8cfcc9e32c31 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:06.943477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 770/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:10.635568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cda131-ab7e-4232-9cc9-c271b7bbf157 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:10.636165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cda131-ab7e-4232-9cc9-c271b7bbf157 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:10.636165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cda131-ab7e-4232-9cc9-c271b7bbf157 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:10.637565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:50:10.643690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d616f-1748-4cce-9434-329d11ad133e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:10.644155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d616f-1748-4cce-9434-329d11ad133e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:10.644272 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d616f-1748-4cce-9434-329d11ad133e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:10.665034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-407d616f-1748-4cce-9434-329d11ad133e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:10.666167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-407d616f-1748-4cce-9434-329d11ad133e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:10.674204 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:10.965397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cda131-ab7e-4232-9cc9-c271b7bbf157 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:10.967864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-84cda131-ab7e-4232-9cc9-c271b7bbf157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:11.047489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cda131-ab7e-4232-9cc9-c271b7bbf157 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:11.051110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47691a8d-fa21-4725-9085-f108ffd5a748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:11.051709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47691a8d-fa21-4725-9085-f108ffd5a748 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:11.051993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47691a8d-fa21-4725-9085-f108ffd5a748 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:11.052273 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47691a8d-fa21-4725-9085-f108ffd5a748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:11.061195 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47691a8d-fa21-4725-9085-f108ffd5a748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:11.063189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47691a8d-fa21-4725-9085-f108ffd5a748 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:11.072486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 759/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:12.992638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc9da1-fe9e-439e-beaa-acec3875f784 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:12.993263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc9da1-fe9e-439e-beaa-acec3875f784 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:12.993263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc9da1-fe9e-439e-beaa-acec3875f784 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:12.994574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:50:12.999876 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c806b2b-195b-4cae-a6b7-555cc65eb783 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:13.000295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c806b2b-195b-4cae-a6b7-555cc65eb783 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:13.000437 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c806b2b-195b-4cae-a6b7-555cc65eb783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:13.021118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-9c806b2b-195b-4cae-a6b7-555cc65eb783 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:13.022328 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9c806b2b-195b-4cae-a6b7-555cc65eb783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:13.030177 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:13.320987 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc9da1-fe9e-439e-beaa-acec3875f784 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:13.322683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2adc9da1-fe9e-439e-beaa-acec3875f784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:13.336415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc9da1-fe9e-439e-beaa-acec3875f784 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:13.339852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2bb3d-1d53-4d55-b72a-5eb97f1ad393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:13.340435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2bb3d-1d53-4d55-b72a-5eb97f1ad393 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:13.340742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2bb3d-1d53-4d55-b72a-5eb97f1ad393 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:13.341016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2bb3d-1d53-4d55-b72a-5eb97f1ad393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:13.349816 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2bb3d-1d53-4d55-b72a-5eb97f1ad393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:13.351251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2bb3d-1d53-4d55-b72a-5eb97f1ad393 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:13.360531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 773/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:16.849719 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f49e45-6b46-4e44-a02c-21ec2c0b9b69 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:16.850314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f49e45-6b46-4e44-a02c-21ec2c0b9b69 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:16.850314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f49e45-6b46-4e44-a02c-21ec2c0b9b69 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:16.851656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:50:16.857057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6861f9-023b-4d8a-8146-fad296d12f1d None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:16.857492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6861f9-023b-4d8a-8146-fad296d12f1d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:16.857595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6861f9-023b-4d8a-8146-fad296d12f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:16.865468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=81 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:16.877832 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9b6861f9-023b-4d8a-8146-fad296d12f1d None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:16.879024 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9b6861f9-023b-4d8a-8146-fad296d12f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:16.887080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:17.435249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f49e45-6b46-4e44-a02c-21ec2c0b9b69 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:17.437554 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-96f49e45-6b46-4e44-a02c-21ec2c0b9b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:17.502722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:17.515765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f49e45-6b46-4e44-a02c-21ec2c0b9b69 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:17.519345 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ae6cd-639c-4e35-a6e4-d47834c16ca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:17.522059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ae6cd-639c-4e35-a6e4-d47834c16ca2 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:17.522059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ae6cd-639c-4e35-a6e4-d47834c16ca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:17.522059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ae6cd-639c-4e35-a6e4-d47834c16ca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:17.529505 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ae6cd-639c-4e35-a6e4-d47834c16ca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:17.531424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ae6cd-639c-4e35-a6e4-d47834c16ca2 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:17.541187 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 762/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:19.460527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d7eb8-69fd-4df4-84a7-1d5953be4ef1 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:19.461149 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d7eb8-69fd-4df4-84a7-1d5953be4ef1 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:19.461149 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d7eb8-69fd-4df4-84a7-1d5953be4ef1 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:19.462465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:50:19.467737 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd01cef-b0d0-4ab7-987e-24b8ca08c83b None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:19.468462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd01cef-b0d0-4ab7-987e-24b8ca08c83b None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:19.468462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd01cef-b0d0-4ab7-987e-24b8ca08c83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:19.487617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-bbd01cef-b0d0-4ab7-987e-24b8ca08c83b None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:19.488856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-bbd01cef-b0d0-4ab7-987e-24b8ca08c83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:19.496698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:20.043230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d7eb8-69fd-4df4-84a7-1d5953be4ef1 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:20.045140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-727d7eb8-69fd-4df4-84a7-1d5953be4ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:20.058243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d7eb8-69fd-4df4-84a7-1d5953be4ef1 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:20.061746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2bc13-2a54-44ee-9c5c-16e3f2687444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:20.062333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2bc13-2a54-44ee-9c5c-16e3f2687444 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:20.062636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2bc13-2a54-44ee-9c5c-16e3f2687444 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:20.062888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2bc13-2a54-44ee-9c5c-16e3f2687444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:20.071520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2bc13-2a54-44ee-9c5c-16e3f2687444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:20.072971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2bc13-2a54-44ee-9c5c-16e3f2687444 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:20.082367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 776/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:29.482794 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=11 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:30.078111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=8 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:43.203163 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dce13-95b3-4b26-a908-d269f2894ddf None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.203825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dce13-95b3-4b26-a908-d269f2894ddf None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.203825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dce13-95b3-4b26-a908-d269f2894ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:43.231298 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-bc7dce13-95b3-4b26-a908-d269f2894ddf None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:43.233644 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-bc7dce13-95b3-4b26-a908-d269f2894ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:43.244494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 764/1540] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:43.254631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec32e07-2036-4f08-b99c-2eb48e700944 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.255233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec32e07-2036-4f08-b99c-2eb48e700944 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.255233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec32e07-2036-4f08-b99c-2eb48e700944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:43.286066 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9ec32e07-2036-4f08-b99c-2eb48e700944 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:43.287434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9ec32e07-2036-4f08-b99c-2eb48e700944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:43.297673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 777/1541] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:43.302949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7dce13-95b3-4b26-a908-d269f2894ddf None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:43.317551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7dce13-95b3-4b26-a908-d269f2894ddf None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:43.325281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53a139-c0a3-42be-b264-a7a3e681091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:43.325870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53a139-c0a3-42be-b264-a7a3e681091e admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.326154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53a139-c0a3-42be-b264-a7a3e681091e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.326432 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53a139-c0a3-42be-b264-a7a3e681091e admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:43.327090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53a139-c0a3-42be-b264-a7a3e681091e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:51:43.327396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53a139-c0a3-42be-b264-a7a3e681091e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:43.329039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53a139-c0a3-42be-b264-a7a3e681091e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:43.347842 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 765/1542] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:51:43.348384 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690074-7dfe-41bf-a8fd-9b5afab53919 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.349180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690074-7dfe-41bf-a8fd-9b5afab53919 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.356878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690074-7dfe-41bf-a8fd-9b5afab53919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:43.357043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53a139-c0a3-42be-b264-a7a3e681091e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:43.370302 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53a139-c0a3-42be-b264-a7a3e681091e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:43.382382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb08da-4250-455a-b7ff-23c59695d5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:43.383011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb08da-4250-455a-b7ff-23c59695d5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.383299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb08da-4250-455a-b7ff-23c59695d5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.383575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb08da-4250-455a-b7ff-23c59695d5b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:43.384204 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb08da-4250-455a-b7ff-23c59695d5b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:43.385269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: WARNING py.warnings [None req-1beb08da-4250-455a-b7ff-23c59695d5b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:51:43.385269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: warnings.warn(msg) Aug 24 07:51:43.385269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: Aug 24 07:51:43.386666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb08da-4250-455a-b7ff-23c59695d5b3 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:43.389881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-e8690074-7dfe-41bf-a8fd-9b5afab53919 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:43.391523 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e8690074-7dfe-41bf-a8fd-9b5afab53919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:43.410426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 778/1543] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:43.419546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bed7e4-59ce-4c7c-8174-b10f6d57741e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.419989 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bed7e4-59ce-4c7c-8174-b10f6d57741e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.420951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bed7e4-59ce-4c7c-8174-b10f6d57741e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:43.461477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-c3bed7e4-59ce-4c7c-8174-b10f6d57741e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:43.463163 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c3bed7e4-59ce-4c7c-8174-b10f6d57741e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:43.482255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 779/1544] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:51:43.487603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bed7e4-59ce-4c7c-8174-b10f6d57741e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:43.499077 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bed7e4-59ce-4c7c-8174-b10f6d57741e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:43.508971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e1df8-ce0f-49b0-bc26-e27a1ce70e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:43.514913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e1df8-ce0f-49b0-bc26-e27a1ce70e85 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.514913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e1df8-ce0f-49b0-bc26-e27a1ce70e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.514913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e1df8-ce0f-49b0-bc26-e27a1ce70e85 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:43.514913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e1df8-ce0f-49b0-bc26-e27a1ce70e85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:51:43.514913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e1df8-ce0f-49b0-bc26-e27a1ce70e85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:43.515357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: WARNING py.warnings [None req-911e1df8-ce0f-49b0-bc26-e27a1ce70e85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:51:43.515357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: warnings.warn(msg) Aug 24 07:51:43.515357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: Aug 24 07:51:43.515357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e1df8-ce0f-49b0-bc26-e27a1ce70e85 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:43.518887 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 766/1545] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:43 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:51:43.523410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb08da-4250-455a-b7ff-23c59695d5b3 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:43.534766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 780/1546] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:51:43.543582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e1df8-ce0f-49b0-bc26-e27a1ce70e85 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:43.558125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e1df8-ce0f-49b0-bc26-e27a1ce70e85 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:43.677216 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5e556-2943-48c0-a2c8-9010b9ab91ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:43.677771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5e556-2943-48c0-a2c8-9010b9ab91ca admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.678058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5e556-2943-48c0-a2c8-9010b9ab91ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.678329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5e556-2943-48c0-a2c8-9010b9ab91ca admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:43.678974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5e556-2943-48c0-a2c8-9010b9ab91ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:43.680913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5e556-2943-48c0-a2c8-9010b9ab91ca admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:43.693495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb08da-4250-455a-b7ff-23c59695d5b3 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:43.715069 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c1b57-8944-4b95-83c6-fe5fa1b8caf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:43.720362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c1b57-8944-4b95-83c6-fe5fa1b8caf8 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.720362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c1b57-8944-4b95-83c6-fe5fa1b8caf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.720362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c1b57-8944-4b95-83c6-fe5fa1b8caf8 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:43.720362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c1b57-8944-4b95-83c6-fe5fa1b8caf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:43.720362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c1b57-8944-4b95-83c6-fe5fa1b8caf8 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:43.823188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 781/1547] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:43 2022] POST /identity/v3/projects => generated 373 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:43.825039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1632729-c7ea-4df5-8196-edc5f76c0bdc None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.825478 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1632729-c7ea-4df5-8196-edc5f76c0bdc None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.826066 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1632729-c7ea-4df5-8196-edc5f76c0bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:43.847702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 767/1548] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:43 2022] POST /identity/v3/users => generated 293 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:51:43.849155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6497f1-1b04-465d-977d-80f6db2782a6 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.849505 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6497f1-1b04-465d-977d-80f6db2782a6 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.849832 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6497f1-1b04-465d-977d-80f6db2782a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:43.859003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-b1632729-c7ea-4df5-8196-edc5f76c0bdc None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:43.860232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b1632729-c7ea-4df5-8196-edc5f76c0bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:43.875765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-da6497f1-1b04-465d-977d-80f6db2782a6 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:43.877159 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-da6497f1-1b04-465d-977d-80f6db2782a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:43.922528 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 782/1549] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:43.923732 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25181fb-f0ec-44cb-bb3b-ff45428b7f7d None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.924091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25181fb-f0ec-44cb-bb3b-ff45428b7f7d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.924297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25181fb-f0ec-44cb-bb3b-ff45428b7f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:43.946505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-e25181fb-f0ec-44cb-bb3b-ff45428b7f7d None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:43.946691 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 768/1550] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:43.946793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e25181fb-f0ec-44cb-bb3b-ff45428b7f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:43.947094 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6497f1-1b04-465d-977d-80f6db2782a6 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:43.955580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 783/1551] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:43.957625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6497f1-1b04-465d-977d-80f6db2782a6 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:43.957973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec4abe-1690-4c6c-821c-55dc612f69f5 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.958266 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec4abe-1690-4c6c-821c-55dc612f69f5 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.958546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec4abe-1690-4c6c-821c-55dc612f69f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:43.962568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca42826-6f7e-4491-a579-719253b2d54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:43.963166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca42826-6f7e-4491-a579-719253b2d54a admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.963448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca42826-6f7e-4491-a579-719253b2d54a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.964115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca42826-6f7e-4491-a579-719253b2d54a admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:43.964462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca42826-6f7e-4491-a579-719253b2d54a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:43.966940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca42826-6f7e-4491-a579-719253b2d54a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:43.979084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-76ec4abe-1690-4c6c-821c-55dc612f69f5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:43.980265 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-76ec4abe-1690-4c6c-821c-55dc612f69f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:43.988956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 784/1552] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:43.990101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eda71a-f217-4991-be67-a8b6f3270731 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:43.990386 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eda71a-f217-4991-be67-a8b6f3270731 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:43.990661 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eda71a-f217-4991-be67-a8b6f3270731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.009341 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-26eda71a-f217-4991-be67-a8b6f3270731 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:44.010509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-26eda71a-f217-4991-be67-a8b6f3270731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:44.018803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 785/1553] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:44.019612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eda71a-f217-4991-be67-a8b6f3270731 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.029627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eda71a-f217-4991-be67-a8b6f3270731 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.034438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92089459-5867-40a8-b5cc-2dffff9afa32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.035014 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92089459-5867-40a8-b5cc-2dffff9afa32 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.035290 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92089459-5867-40a8-b5cc-2dffff9afa32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.035561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92089459-5867-40a8-b5cc-2dffff9afa32 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.036193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92089459-5867-40a8-b5cc-2dffff9afa32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.037808 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92089459-5867-40a8-b5cc-2dffff9afa32 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.048388 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 786/1554] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:44.049614 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc70e0-065e-4e69-b147-57cabb003bfd None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.049903 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc70e0-065e-4e69-b147-57cabb003bfd None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.050178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc70e0-065e-4e69-b147-57cabb003bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.063272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 769/1555] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:43 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:44.069005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8deb94e-7de9-46da-a0d4-d179d60b3a05 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.069005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8deb94e-7de9-46da-a0d4-d179d60b3a05 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.069005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8deb94e-7de9-46da-a0d4-d179d60b3a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.070490 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-50dc70e0-065e-4e69-b147-57cabb003bfd None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:44.071768 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-50dc70e0-065e-4e69-b147-57cabb003bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:44.084726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 787/1556] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:44.084726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019600f-6cb6-4f1c-b410-f524c13c40d5 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.084726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019600f-6cb6-4f1c-b410-f524c13c40d5 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.084726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019600f-6cb6-4f1c-b410-f524c13c40d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.086290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-c8deb94e-7de9-46da-a0d4-d179d60b3a05 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:44.087582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c8deb94e-7de9-46da-a0d4-d179d60b3a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:44.097015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 770/1557] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:44.098262 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bbc3d-8e57-4b96-9762-4c32c4086072 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.098679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bbc3d-8e57-4b96-9762-4c32c4086072 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.098960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bbc3d-8e57-4b96-9762-4c32c4086072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.109083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9019600f-6cb6-4f1c-b410-f524c13c40d5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:44.110329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9019600f-6cb6-4f1c-b410-f524c13c40d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:44.118584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 788/1558] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:44.119050 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d90bbc3d-8e57-4b96-9762-4c32c4086072 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:44.119833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b953a1-7744-489c-958e-8ba14f119dff None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.120126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b953a1-7744-489c-958e-8ba14f119dff None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.120372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d90bbc3d-8e57-4b96-9762-4c32c4086072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:44.120492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b953a1-7744-489c-958e-8ba14f119dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.129411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 771/1559] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:44.130739 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa14cb-a563-4cd9-a4c0-cceeefc747ec None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.131028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa14cb-a563-4cd9-a4c0-cceeefc747ec None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.131299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa14cb-a563-4cd9-a4c0-cceeefc747ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.139705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-d5b953a1-7744-489c-958e-8ba14f119dff None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:44.140939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d5b953a1-7744-489c-958e-8ba14f119dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:44.148669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 789/1560] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:44.149955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-8dfa14cb-a563-4cd9-a4c0-cceeefc747ec None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:44.150273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b953a1-7744-489c-958e-8ba14f119dff None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.151141 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-8dfa14cb-a563-4cd9-a4c0-cceeefc747ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:44.155564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:44.159971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 772/1561] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:44.161818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa14cb-a563-4cd9-a4c0-cceeefc747ec None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.161910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b953a1-7744-489c-958e-8ba14f119dff None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.169213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:44.169338 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f6201-ded7-4fd3-a24a-f7bada3ac2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.169338 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f6201-ded7-4fd3-a24a-f7bada3ac2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.169338 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f6201-ded7-4fd3-a24a-f7bada3ac2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.169338 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f6201-ded7-4fd3-a24a-f7bada3ac2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.170252 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f6201-ded7-4fd3-a24a-f7bada3ac2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.170252 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f6201-ded7-4fd3-a24a-f7bada3ac2e6 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.171376 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa14cb-a563-4cd9-a4c0-cceeefc747ec None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.176118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c861a-af42-40d6-ad53-17c71e9c1614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.176747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c861a-af42-40d6-ad53-17c71e9c1614 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.177023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c861a-af42-40d6-ad53-17c71e9c1614 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.177294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c861a-af42-40d6-ad53-17c71e9c1614 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.177925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c861a-af42-40d6-ad53-17c71e9c1614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.179504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c861a-af42-40d6-ad53-17c71e9c1614 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.184959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 790/1562] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:44.184959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fc8e9-1df0-4cb9-9d08-b609869fddb1 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.184959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fc8e9-1df0-4cb9-9d08-b609869fddb1 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.184959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fc8e9-1df0-4cb9-9d08-b609869fddb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.189731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 773/1563] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:44.190538 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c861a-af42-40d6-ad53-17c71e9c1614 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.201731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c861a-af42-40d6-ad53-17c71e9c1614 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.206364 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-7b6fc8e9-1df0-4cb9-9d08-b609869fddb1 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:44.206541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd432f0-a7f7-489f-b4aa-96bad71adf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.207129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd432f0-a7f7-489f-b4aa-96bad71adf22 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.207411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd432f0-a7f7-489f-b4aa-96bad71adf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.207684 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd432f0-a7f7-489f-b4aa-96bad71adf22 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.207827 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7b6fc8e9-1df0-4cb9-9d08-b609869fddb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:44.208341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd432f0-a7f7-489f-b4aa-96bad71adf22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:51:44.208662 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd432f0-a7f7-489f-b4aa-96bad71adf22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.210358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd432f0-a7f7-489f-b4aa-96bad71adf22 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.215844 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 791/1564] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:44.217372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fc8e9-1df0-4cb9-9d08-b609869fddb1 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.224042 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 774/1565] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:51:44.226627 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd432f0-a7f7-489f-b4aa-96bad71adf22 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.226745 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fc8e9-1df0-4cb9-9d08-b609869fddb1 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.231604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd01e9d-a1d1-4b2f-87aa-f55a19143a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.232771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd01e9d-a1d1-4b2f-87aa-f55a19143a03 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.233142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd01e9d-a1d1-4b2f-87aa-f55a19143a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.233423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd01e9d-a1d1-4b2f-87aa-f55a19143a03 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.234068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd01e9d-a1d1-4b2f-87aa-f55a19143a03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:51:44.234372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd01e9d-a1d1-4b2f-87aa-f55a19143a03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.235976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd01e9d-a1d1-4b2f-87aa-f55a19143a03 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.236087 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd432f0-a7f7-489f-b4aa-96bad71adf22 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.240722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344385dd-dd62-489e-83b9-853fc5a51492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.241312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344385dd-dd62-489e-83b9-853fc5a51492 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.241621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344385dd-dd62-489e-83b9-853fc5a51492 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.241897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344385dd-dd62-489e-83b9-853fc5a51492 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.242559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344385dd-dd62-489e-83b9-853fc5a51492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:51:44.243042 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344385dd-dd62-489e-83b9-853fc5a51492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.244621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344385dd-dd62-489e-83b9-853fc5a51492 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.248339 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 792/1566] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:51:44.249186 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd01e9d-a1d1-4b2f-87aa-f55a19143a03 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.263191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 775/1567] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:51:44.263191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344385dd-dd62-489e-83b9-853fc5a51492 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.264145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd01e9d-a1d1-4b2f-87aa-f55a19143a03 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.267002 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344385dd-dd62-489e-83b9-853fc5a51492 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.268989 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c8825-ae90-4b8e-a5ff-26ffb067e08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.269587 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c8825-ae90-4b8e-a5ff-26ffb067e08e admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.269867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c8825-ae90-4b8e-a5ff-26ffb067e08e admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.270138 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c8825-ae90-4b8e-a5ff-26ffb067e08e admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.270779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c8825-ae90-4b8e-a5ff-26ffb067e08e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:51:44.271072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c8825-ae90-4b8e-a5ff-26ffb067e08e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.271593 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e622ebe-0620-4a78-a97e-5545702c6261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.272186 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e622ebe-0620-4a78-a97e-5545702c6261 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.272478 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e622ebe-0620-4a78-a97e-5545702c6261 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.272815 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c8825-ae90-4b8e-a5ff-26ffb067e08e admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.273005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e622ebe-0620-4a78-a97e-5545702c6261 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.273815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e622ebe-0620-4a78-a97e-5545702c6261 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:51:44.274650 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e622ebe-0620-4a78-a97e-5545702c6261 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.276230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e622ebe-0620-4a78-a97e-5545702c6261 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.285496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 793/1568] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:51:44.286278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c8825-ae90-4b8e-a5ff-26ffb067e08e admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.302056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 776/1569] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:51:44.302251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c8825-ae90-4b8e-a5ff-26ffb067e08e admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.302370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e622ebe-0620-4a78-a97e-5545702c6261 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.306339 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54715871-dc5f-48ff-83ce-85fb0d7de61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.307030 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54715871-dc5f-48ff-83ce-85fb0d7de61a admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.307311 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54715871-dc5f-48ff-83ce-85fb0d7de61a admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.307587 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54715871-dc5f-48ff-83ce-85fb0d7de61a admin admin] PATH_INFO: `/v3/projects/ee7ef4f035fd402fb7e21a21c40e7c49/users/aee01beeba5b48c3b6c25489743de40e/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.307701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e622ebe-0620-4a78-a97e-5545702c6261 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.312494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af922ff-97dc-4556-8d54-b11ec7ef1c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.313129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af922ff-97dc-4556-8d54-b11ec7ef1c11 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.313526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af922ff-97dc-4556-8d54-b11ec7ef1c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.313800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af922ff-97dc-4556-8d54-b11ec7ef1c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.314438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af922ff-97dc-4556-8d54-b11ec7ef1c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.316089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af922ff-97dc-4556-8d54-b11ec7ef1c11 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.322166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54715871-dc5f-48ff-83ce-85fb0d7de61a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7ef4f035fd402fb7e21a21c40e7c49, user_id=aee01beeba5b48c3b6c25489743de40e, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.324838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54715871-dc5f-48ff-83ce-85fb0d7de61a admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.326616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 777/1570] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:44.327410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af922ff-97dc-4556-8d54-b11ec7ef1c11 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.336931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af922ff-97dc-4556-8d54-b11ec7ef1c11 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.341460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e562c-8d47-4ab8-b555-0de37f78b8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.342020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e562c-8d47-4ab8-b555-0de37f78b8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.344179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e562c-8d47-4ab8-b555-0de37f78b8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.344179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e562c-8d47-4ab8-b555-0de37f78b8a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.344179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e562c-8d47-4ab8-b555-0de37f78b8a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:51:44.344179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e562c-8d47-4ab8-b555-0de37f78b8a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.345072 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e562c-8d47-4ab8-b555-0de37f78b8a5 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.356759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 778/1571] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:51:44.358026 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e562c-8d47-4ab8-b555-0de37f78b8a5 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.368059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e562c-8d47-4ab8-b555-0de37f78b8a5 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.373466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fc8ec-a5b3-4cc4-ae67-c6bc79a4c486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.374060 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fc8ec-a5b3-4cc4-ae67-c6bc79a4c486 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.377594 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fc8ec-a5b3-4cc4-ae67-c6bc79a4c486 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.377594 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fc8ec-a5b3-4cc4-ae67-c6bc79a4c486 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.377594 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fc8ec-a5b3-4cc4-ae67-c6bc79a4c486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.377594 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fc8ec-a5b3-4cc4-ae67-c6bc79a4c486 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.399838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 794/1572] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:44 2022] PUT /identity/v3/projects/ee7ef4f035fd402fb7e21a21c40e7c49/users/aee01beeba5b48c3b6c25489743de40e/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:44.403069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54715871-dc5f-48ff-83ce-85fb0d7de61a admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.459028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 779/1573] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:44.460057 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fc8ec-a5b3-4cc4-ae67-c6bc79a4c486 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.461563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54715871-dc5f-48ff-83ce-85fb0d7de61a admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.518314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b375b0-d66d-4b9d-98f0-4cce5c47aa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.518919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b375b0-d66d-4b9d-98f0-4cce5c47aa6b admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.519201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b375b0-d66d-4b9d-98f0-4cce5c47aa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.519476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b375b0-d66d-4b9d-98f0-4cce5c47aa6b admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.520100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b375b0-d66d-4b9d-98f0-4cce5c47aa6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.522073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b375b0-d66d-4b9d-98f0-4cce5c47aa6b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.523387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fc8ec-a5b3-4cc4-ae67-c6bc79a4c486 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.529058 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f806f04-a83d-4a6d-ad24-b4289affab40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.529682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f806f04-a83d-4a6d-ad24-b4289affab40 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.529968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f806f04-a83d-4a6d-ad24-b4289affab40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.530247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f806f04-a83d-4a6d-ad24-b4289affab40 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.530883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f806f04-a83d-4a6d-ad24-b4289affab40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.533744 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f806f04-a83d-4a6d-ad24-b4289affab40 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.613036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 795/1574] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/projects => generated 391 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:44.613955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b375b0-d66d-4b9d-98f0-4cce5c47aa6b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.635437 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 780/1575] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:44.640280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f806f04-a83d-4a6d-ad24-b4289affab40 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.647600 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:44.680323 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b375b0-d66d-4b9d-98f0-4cce5c47aa6b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.687783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:44.698524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f806f04-a83d-4a6d-ad24-b4289affab40 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.703630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec6efa-c204-41cb-8585-f55bacf724d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.704234 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec6efa-c204-41cb-8585-f55bacf724d2 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.704512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec6efa-c204-41cb-8585-f55bacf724d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.704821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec6efa-c204-41cb-8585-f55bacf724d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.705695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec6efa-c204-41cb-8585-f55bacf724d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.707623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec6efa-c204-41cb-8585-f55bacf724d2 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.738666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084257a-6865-4f81-ae95-3cc226e9a2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.739212 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084257a-6865-4f81-ae95-3cc226e9a2b7 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.739499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084257a-6865-4f81-ae95-3cc226e9a2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.739777 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084257a-6865-4f81-ae95-3cc226e9a2b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.740401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084257a-6865-4f81-ae95-3cc226e9a2b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.742380 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084257a-6865-4f81-ae95-3cc226e9a2b7 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.784228 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 781/1576] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:44.785174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec6efa-c204-41cb-8585-f55bacf724d2 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.799543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 796/1577] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/projects => generated 377 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:44.799543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084257a-6865-4f81-ae95-3cc226e9a2b7 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:44.846621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec6efa-c204-41cb-8585-f55bacf724d2 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.859147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084257a-6865-4f81-ae95-3cc226e9a2b7 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:44.863910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac555c1f-3be4-4dd7-99f6-bacc71e8817e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.864512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac555c1f-3be4-4dd7-99f6-bacc71e8817e admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.864825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac555c1f-3be4-4dd7-99f6-bacc71e8817e admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.865098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac555c1f-3be4-4dd7-99f6-bacc71e8817e admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.865720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac555c1f-3be4-4dd7-99f6-bacc71e8817e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.867626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac555c1f-3be4-4dd7-99f6-bacc71e8817e admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.901289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99410641-3a82-4d83-bcbb-85885a248975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:44.901947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99410641-3a82-4d83-bcbb-85885a248975 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.902229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99410641-3a82-4d83-bcbb-85885a248975 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.902500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99410641-3a82-4d83-bcbb-85885a248975 admin admin] PATH_INFO: `/v3/projects/8e0daa1f56f4432d880f2eb4b85391fb/users/39878b432b9c42a9a0e70f72cb010d75/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.917095 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99410641-3a82-4d83-bcbb-85885a248975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e0daa1f56f4432d880f2eb4b85391fb, user_id=39878b432b9c42a9a0e70f72cb010d75, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:44.918116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: WARNING py.warnings [None req-99410641-3a82-4d83-bcbb-85885a248975 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:51:44.918116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: warnings.warn(msg) Aug 24 07:51:44.918116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: Aug 24 07:51:44.920752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99410641-3a82-4d83-bcbb-85885a248975 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:44.937863 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 797/1578] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:44.939897 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57311299-f67f-47ce-af07-70fc23dbb2a1 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:44.940184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57311299-f67f-47ce-af07-70fc23dbb2a1 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:44.940481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57311299-f67f-47ce-af07-70fc23dbb2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:44.959028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-57311299-f67f-47ce-af07-70fc23dbb2a1 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:44.960211 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-57311299-f67f-47ce-af07-70fc23dbb2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:44.994533 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 782/1579] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:44 2022] PUT /identity/v3/projects/8e0daa1f56f4432d880f2eb4b85391fb/users/39878b432b9c42a9a0e70f72cb010d75/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:44.995400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99410641-3a82-4d83-bcbb-85885a248975 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.008479 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:45.027909 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 798/1580] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:45.029263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57311299-f67f-47ce-af07-70fc23dbb2a1 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.039529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57311299-f67f-47ce-af07-70fc23dbb2a1 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.062251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99410641-3a82-4d83-bcbb-85885a248975 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.067089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bfd2f-7100-44ce-afbc-37cd46f17a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.067699 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bfd2f-7100-44ce-afbc-37cd46f17a9d admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.067983 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bfd2f-7100-44ce-afbc-37cd46f17a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.068262 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bfd2f-7100-44ce-afbc-37cd46f17a9d admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.068926 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bfd2f-7100-44ce-afbc-37cd46f17a9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.070849 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bfd2f-7100-44ce-afbc-37cd46f17a9d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.082964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:45.099330 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dc37e-7076-4105-8320-952bb928334b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.099938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dc37e-7076-4105-8320-952bb928334b admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.100221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dc37e-7076-4105-8320-952bb928334b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.100515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dc37e-7076-4105-8320-952bb928334b admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.101214 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dc37e-7076-4105-8320-952bb928334b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.102231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: WARNING py.warnings [None req-0a8dc37e-7076-4105-8320-952bb928334b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:51:45.102231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: warnings.warn(msg) Aug 24 07:51:45.102231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: Aug 24 07:51:45.103419 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dc37e-7076-4105-8320-952bb928334b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.148235 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 783/1581] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:44 2022] POST /identity/v3/users => generated 284 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:45.149202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bfd2f-7100-44ce-afbc-37cd46f17a9d admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.159556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bfd2f-7100-44ce-afbc-37cd46f17a9d admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.164171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb860f-25bf-4ad7-a6da-b61eb136f2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.164787 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb860f-25bf-4ad7-a6da-b61eb136f2ff admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.165073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb860f-25bf-4ad7-a6da-b61eb136f2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.165369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb860f-25bf-4ad7-a6da-b61eb136f2ff admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.166205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb860f-25bf-4ad7-a6da-b61eb136f2ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.168121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb860f-25bf-4ad7-a6da-b61eb136f2ff admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.211242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 799/1582] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:45 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:45.212045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dc37e-7076-4105-8320-952bb928334b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.221758 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dc37e-7076-4105-8320-952bb928334b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.226306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838f2d7-1cbd-4b4b-a41c-ab48019d4fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.227006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838f2d7-1cbd-4b4b-a41c-ab48019d4fba admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.227290 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838f2d7-1cbd-4b4b-a41c-ab48019d4fba admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.227565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838f2d7-1cbd-4b4b-a41c-ab48019d4fba admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.228170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838f2d7-1cbd-4b4b-a41c-ab48019d4fba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.230109 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838f2d7-1cbd-4b4b-a41c-ab48019d4fba admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.285375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 784/1583] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:45 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:45.286115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb860f-25bf-4ad7-a6da-b61eb136f2ff admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.296076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb860f-25bf-4ad7-a6da-b61eb136f2ff admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.300774 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c07e2-24f6-4adc-bf37-173cd1b09c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.301301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c07e2-24f6-4adc-bf37-173cd1b09c56 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.301615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c07e2-24f6-4adc-bf37-173cd1b09c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.301801 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c07e2-24f6-4adc-bf37-173cd1b09c56 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.302416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c07e2-24f6-4adc-bf37-173cd1b09c56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.304333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c07e2-24f6-4adc-bf37-173cd1b09c56 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.331544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 800/1584] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:45 2022] POST /identity/v3/users => generated 282 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:45.332406 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838f2d7-1cbd-4b4b-a41c-ab48019d4fba admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.341990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838f2d7-1cbd-4b4b-a41c-ab48019d4fba admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.346603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bc50f-298a-4001-b27b-3a0cf40a95e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.347230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bc50f-298a-4001-b27b-3a0cf40a95e7 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.347511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bc50f-298a-4001-b27b-3a0cf40a95e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.347791 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bc50f-298a-4001-b27b-3a0cf40a95e7 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.348393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bc50f-298a-4001-b27b-3a0cf40a95e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.350344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bc50f-298a-4001-b27b-3a0cf40a95e7 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.391412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 785/1585] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:45 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:45.391412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14339ad-3b80-4d92-b06b-315d2c8d7bd1 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.391412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14339ad-3b80-4d92-b06b-315d2c8d7bd1 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.391412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14339ad-3b80-4d92-b06b-315d2c8d7bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.417157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-a14339ad-3b80-4d92-b06b-315d2c8d7bd1 None None] MFA Rules not processed for user `39878b432b9c42a9a0e70f72cb010d75`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:45.418780 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-a14339ad-3b80-4d92-b06b-315d2c8d7bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:45.446647 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 801/1586] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:45 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:45.447629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bc50f-298a-4001-b27b-3a0cf40a95e7 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.489190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 786/1587] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:45.491066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14339ad-3b80-4d92-b06b-315d2c8d7bd1 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.502667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14339ad-3b80-4d92-b06b-315d2c8d7bd1 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.505724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bc50f-298a-4001-b27b-3a0cf40a95e7 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.507184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e614379-6aba-4947-b744-a617878fab5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.507932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e614379-6aba-4947-b744-a617878fab5e admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.508217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e614379-6aba-4947-b744-a617878fab5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.508544 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e614379-6aba-4947-b744-a617878fab5e admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.509261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e614379-6aba-4947-b744-a617878fab5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.509791 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d4c21-b0b3-4a70-a857-aeba895c2243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.510396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d4c21-b0b3-4a70-a857-aeba895c2243 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.510679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d4c21-b0b3-4a70-a857-aeba895c2243 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.510925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e614379-6aba-4947-b744-a617878fab5e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.511032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d4c21-b0b3-4a70-a857-aeba895c2243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.525810 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 787/1588] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:45.526002 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d4c21-b0b3-4a70-a857-aeba895c2243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.526104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e614379-6aba-4947-b744-a617878fab5e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.526185 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d4c21-b0b3-4a70-a857-aeba895c2243 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.544712 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e614379-6aba-4947-b744-a617878fab5e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.544965 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 802/1589] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:45.544965 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d4c21-b0b3-4a70-a857-aeba895c2243 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.545384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffefa97-cc39-4bf5-9f46-48df23a982ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.545384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffefa97-cc39-4bf5-9f46-48df23a982ab admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.545384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffefa97-cc39-4bf5-9f46-48df23a982ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.545384 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffefa97-cc39-4bf5-9f46-48df23a982ab admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.545764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffefa97-cc39-4bf5-9f46-48df23a982ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.546572 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffefa97-cc39-4bf5-9f46-48df23a982ab admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.550930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d4c21-b0b3-4a70-a857-aeba895c2243 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.556080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad06415-a173-47a4-a358-1d656ede11f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.556354 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad06415-a173-47a4-a358-1d656ede11f4 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.556637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad06415-a173-47a4-a358-1d656ede11f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.557114 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad06415-a173-47a4-a358-1d656ede11f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.557238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 788/1590] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:45.557781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffefa97-cc39-4bf5-9f46-48df23a982ab admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.561288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad06415-a173-47a4-a358-1d656ede11f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.562950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad06415-a173-47a4-a358-1d656ede11f4 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.569594 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffefa97-cc39-4bf5-9f46-48df23a982ab admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.573533 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 803/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:45.576598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6ea9e-1e6a-4ab9-a452-3e70412b9871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.576801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad06415-a173-47a4-a358-1d656ede11f4 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.576897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6ea9e-1e6a-4ab9-a452-3e70412b9871 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.576897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6ea9e-1e6a-4ab9-a452-3e70412b9871 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.576897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6ea9e-1e6a-4ab9-a452-3e70412b9871 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.577123 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6ea9e-1e6a-4ab9-a452-3e70412b9871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.578739 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6ea9e-1e6a-4ab9-a452-3e70412b9871 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.585405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad06415-a173-47a4-a358-1d656ede11f4 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.593447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 789/1592] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:45.593596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5c093-4a70-42ec-b215-7686f5aa69b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.593731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6ea9e-1e6a-4ab9-a452-3e70412b9871 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.593824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5c093-4a70-42ec-b215-7686f5aa69b1 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.593824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5c093-4a70-42ec-b215-7686f5aa69b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.593824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5c093-4a70-42ec-b215-7686f5aa69b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.593824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5c093-4a70-42ec-b215-7686f5aa69b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.597951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5c093-4a70-42ec-b215-7686f5aa69b1 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.603772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6ea9e-1e6a-4ab9-a452-3e70412b9871 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.611984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 804/1593] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:45.612125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4864d04-cecf-446c-877c-32e4031eaf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.612125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4864d04-cecf-446c-877c-32e4031eaf88 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.612125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4864d04-cecf-446c-877c-32e4031eaf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.612413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad629a5-c0dd-4621-a2a6-04d9ec114681 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.612506 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4864d04-cecf-446c-877c-32e4031eaf88 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.612601 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad629a5-c0dd-4621-a2a6-04d9ec114681 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.612601 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad629a5-c0dd-4621-a2a6-04d9ec114681 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.612818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4864d04-cecf-446c-877c-32e4031eaf88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.612906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 805/1594] 10.222.0.26 () {54 vars in 962 bytes} [Wed Aug 24 07:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:51:45.613345 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad629a5-c0dd-4621-a2a6-04d9ec114681 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.617803 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4864d04-cecf-446c-877c-32e4031eaf88 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.625918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad629a5-c0dd-4621-a2a6-04d9ec114681 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.628566 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 790/1595] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:45.630100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4864d04-cecf-446c-877c-32e4031eaf88 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.630880 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bf120-1808-4131-8fbe-298fddbe9361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.631561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bf120-1808-4131-8fbe-298fddbe9361 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.631902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bf120-1808-4131-8fbe-298fddbe9361 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.632215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bf120-1808-4131-8fbe-298fddbe9361 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.632898 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bf120-1808-4131-8fbe-298fddbe9361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.634488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bf120-1808-4131-8fbe-298fddbe9361 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.641567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4864d04-cecf-446c-877c-32e4031eaf88 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.646033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 806/1596] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:45.646742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bf120-1808-4131-8fbe-298fddbe9361 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.646986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c53434-5e04-42ba-8a14-1967fa8e1995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.647628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c53434-5e04-42ba-8a14-1967fa8e1995 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.647912 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c53434-5e04-42ba-8a14-1967fa8e1995 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.648189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c53434-5e04-42ba-8a14-1967fa8e1995 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.649206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c53434-5e04-42ba-8a14-1967fa8e1995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.650899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c53434-5e04-42ba-8a14-1967fa8e1995 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.659048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bf120-1808-4131-8fbe-298fddbe9361 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.662347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 791/1597] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:45.664524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc6b4b-5ff6-4365-85cd-c04421b0dbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.665195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc6b4b-5ff6-4365-85cd-c04421b0dbda admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.665481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc6b4b-5ff6-4365-85cd-c04421b0dbda admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.665756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc6b4b-5ff6-4365-85cd-c04421b0dbda admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.666458 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc6b4b-5ff6-4365-85cd-c04421b0dbda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.666621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c53434-5e04-42ba-8a14-1967fa8e1995 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.668133 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc6b4b-5ff6-4365-85cd-c04421b0dbda admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.672000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:45.678008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c53434-5e04-42ba-8a14-1967fa8e1995 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.686516 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 807/1598] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:45.686676 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b80df-7d20-48f1-9ad8-9f90cd4a4f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.686808 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc6b4b-5ff6-4365-85cd-c04421b0dbda admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.686907 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b80df-7d20-48f1-9ad8-9f90cd4a4f9a admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.686907 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b80df-7d20-48f1-9ad8-9f90cd4a4f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.686907 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b80df-7d20-48f1-9ad8-9f90cd4a4f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.686907 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b80df-7d20-48f1-9ad8-9f90cd4a4f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.688022 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b80df-7d20-48f1-9ad8-9f90cd4a4f9a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.698652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc6b4b-5ff6-4365-85cd-c04421b0dbda admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.702254 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 792/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:45.702254 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89088a04-7439-4f05-ba4f-794dd494c3f8 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.702254 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89088a04-7439-4f05-ba4f-794dd494c3f8 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.702254 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89088a04-7439-4f05-ba4f-794dd494c3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.706825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3437c-cf09-46a0-a478-7d4ebba19b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.707537 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3437c-cf09-46a0-a478-7d4ebba19b65 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.707874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3437c-cf09-46a0-a478-7d4ebba19b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.708217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3437c-cf09-46a0-a478-7d4ebba19b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.708976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3437c-cf09-46a0-a478-7d4ebba19b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.714472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3437c-cf09-46a0-a478-7d4ebba19b65 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.722998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 808/1600] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:45.724173 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3437c-cf09-46a0-a478-7d4ebba19b65 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.738378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3437c-cf09-46a0-a478-7d4ebba19b65 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.743127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f6623-04fe-4c4a-8feb-870f1314fb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.743818 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f6623-04fe-4c4a-8feb-870f1314fb6a admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.744176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f6623-04fe-4c4a-8feb-870f1314fb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.744527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f6623-04fe-4c4a-8feb-870f1314fb6a admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.744866 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-89088a04-7439-4f05-ba4f-794dd494c3f8 None None] MFA Rules not processed for user `bdf81f09006a42299c0807f77f16c748`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:45.745340 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f6623-04fe-4c4a-8feb-870f1314fb6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.746095 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-89088a04-7439-4f05-ba4f-794dd494c3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:45.747190 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f6623-04fe-4c4a-8feb-870f1314fb6a admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.758836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 809/1601] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:45.759616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f6623-04fe-4c4a-8feb-870f1314fb6a admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.770280 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f6623-04fe-4c4a-8feb-870f1314fb6a admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.774447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:45.775374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05740dc1-32e1-461a-b725-8c60639cac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.776027 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05740dc1-32e1-461a-b725-8c60639cac10 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.776464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05740dc1-32e1-461a-b725-8c60639cac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.776780 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05740dc1-32e1-461a-b725-8c60639cac10 admin admin] PATH_INFO: `/v3/projects/b0202ce802094885ac4e7a24c992b166/users/ee9f4acd6ad247b99ac8a55600c701ff/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.792200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05740dc1-32e1-461a-b725-8c60639cac10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0202ce802094885ac4e7a24c992b166, user_id=ee9f4acd6ad247b99ac8a55600c701ff, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.799043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05740dc1-32e1-461a-b725-8c60639cac10 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.815419 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 793/1602] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Aug 24 07:51:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:45.816109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89088a04-7439-4f05-ba4f-794dd494c3f8 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.826500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89088a04-7439-4f05-ba4f-794dd494c3f8 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:45.831403 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c9260-ac95-4c41-94c4-f02acc6ebc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:45.832031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c9260-ac95-4c41-94c4-f02acc6ebc7d admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:45.832334 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c9260-ac95-4c41-94c4-f02acc6ebc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:45.832633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c9260-ac95-4c41-94c4-f02acc6ebc7d admin admin] PATH_INFO: `/v3/projects/a75ba813beae4999b9ae2afd58de8d25/users/e23edadcfe3d4a0b81b13e278682ca2f/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:45.846628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c9260-ac95-4c41-94c4-f02acc6ebc7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a75ba813beae4999b9ae2afd58de8d25, user_id=e23edadcfe3d4a0b81b13e278682ca2f, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:45.849364 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c9260-ac95-4c41-94c4-f02acc6ebc7d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:45.898883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 810/1603] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:45 2022] PUT /identity/v3/projects/b0202ce802094885ac4e7a24c992b166/users/ee9f4acd6ad247b99ac8a55600c701ff/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:45.898883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05740dc1-32e1-461a-b725-8c60639cac10 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.939325 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 794/1604] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:45 2022] PUT /identity/v3/projects/a75ba813beae4999b9ae2afd58de8d25/users/e23edadcfe3d4a0b81b13e278682ca2f/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:45.943362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c9260-ac95-4c41-94c4-f02acc6ebc7d admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:45.964575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05740dc1-32e1-461a-b725-8c60639cac10 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:46.004541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c9260-ac95-4c41-94c4-f02acc6ebc7d admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:46.009528 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6beba-f99d-4dae-abe4-d87c74dd1dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:46.010201 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6beba-f99d-4dae-abe4-d87c74dd1dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.010469 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6beba-f99d-4dae-abe4-d87c74dd1dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.010746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6beba-f99d-4dae-abe4-d87c74dd1dc1 admin admin] PATH_INFO: `/v3/projects/5a81a64cb57941548f25273bca00ec25/users/19d048358434468c8e48264c10f9e033/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.024033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d9555-5c0e-468b-b4cc-8ee7f12ef48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:46.024732 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d9555-5c0e-468b-b4cc-8ee7f12ef48f admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.025014 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d9555-5c0e-468b-b4cc-8ee7f12ef48f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.025294 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d9555-5c0e-468b-b4cc-8ee7f12ef48f admin admin] PATH_INFO: `/v3/projects/cfedc6a60ab54b75bf9a619ed8369c23/users/fd15e9fec2af4261990e3b6bcb5dea8c/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.026428 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6beba-f99d-4dae-abe4-d87c74dd1dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a81a64cb57941548f25273bca00ec25, user_id=19d048358434468c8e48264c10f9e033, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:46.029203 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6beba-f99d-4dae-abe4-d87c74dd1dc1 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:46.039593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d9555-5c0e-468b-b4cc-8ee7f12ef48f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfedc6a60ab54b75bf9a619ed8369c23, user_id=fd15e9fec2af4261990e3b6bcb5dea8c, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:46.042225 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d9555-5c0e-468b-b4cc-8ee7f12ef48f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:46.148779 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 795/1605] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:45 2022] PUT /identity/v3/projects/5a81a64cb57941548f25273bca00ec25/users/19d048358434468c8e48264c10f9e033/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:46.149901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6beba-f99d-4dae-abe4-d87c74dd1dc1 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:46.175435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 811/1606] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:45 2022] PUT /identity/v3/projects/cfedc6a60ab54b75bf9a619ed8369c23/users/fd15e9fec2af4261990e3b6bcb5dea8c/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:46.175435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d9555-5c0e-468b-b4cc-8ee7f12ef48f admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:46.226760 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6beba-f99d-4dae-abe4-d87c74dd1dc1 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:46.236675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:46.254058 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:46.254228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d9555-5c0e-468b-b4cc-8ee7f12ef48f admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:46.259161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d7e42-797c-4eec-8ad4-a4c0194705c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:46.259829 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7e42-797c-4eec-8ad4-a4c0194705c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.260112 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7e42-797c-4eec-8ad4-a4c0194705c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.260389 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7e42-797c-4eec-8ad4-a4c0194705c0 admin admin] PATH_INFO: `/v3/projects/d7043fd37a1040c2af5d3f9f2943a791/users/2ec3daa674814619976aead75459e98c/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.274855 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d7e42-797c-4eec-8ad4-a4c0194705c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7043fd37a1040c2af5d3f9f2943a791, user_id=2ec3daa674814619976aead75459e98c, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:46.277580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d7e42-797c-4eec-8ad4-a4c0194705c0 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:46.286904 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6825a49-bf53-4831-9722-9b4379d825ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:46.287559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6825a49-bf53-4831-9722-9b4379d825ad admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.287839 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6825a49-bf53-4831-9722-9b4379d825ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.288118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6825a49-bf53-4831-9722-9b4379d825ad admin admin] PATH_INFO: `/v3/projects/af7797a0a97b4debb3492e864b6ca1b1/users/a610194aaf534d3aabb56c9443b5d56d/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.302502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6825a49-bf53-4831-9722-9b4379d825ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=af7797a0a97b4debb3492e864b6ca1b1, user_id=a610194aaf534d3aabb56c9443b5d56d, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:46.305169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6825a49-bf53-4831-9722-9b4379d825ad admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:46.395126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 812/1607] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:46 2022] PUT /identity/v3/projects/d7043fd37a1040c2af5d3f9f2943a791/users/2ec3daa674814619976aead75459e98c/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:46.396067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d7e42-797c-4eec-8ad4-a4c0194705c0 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:46.408018 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 796/1608] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:46 2022] PUT /identity/v3/projects/af7797a0a97b4debb3492e864b6ca1b1/users/a610194aaf534d3aabb56c9443b5d56d/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:46.412968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ca916-a908-4214-953f-966fceb1b96d None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.413331 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ca916-a908-4214-953f-966fceb1b96d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.413684 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ca916-a908-4214-953f-966fceb1b96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.443673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e55ca916-a908-4214-953f-966fceb1b96d None None] MFA Rules not processed for user `ee9f4acd6ad247b99ac8a55600c701ff`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:46.444946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e55ca916-a908-4214-953f-966fceb1b96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:46.476371 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d7e42-797c-4eec-8ad4-a4c0194705c0 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:46.517411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 797/1609] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:46.518735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f07f17-afe0-4cfa-8803-64dd01192c32 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.519019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f07f17-afe0-4cfa-8803-64dd01192c32 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.519294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f07f17-afe0-4cfa-8803-64dd01192c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.541234 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-64f07f17-afe0-4cfa-8803-64dd01192c32 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:46.542426 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-64f07f17-afe0-4cfa-8803-64dd01192c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:46.548147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a7c71-adba-445c-b340-1ef817267e18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:46.548865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a7c71-adba-445c-b340-1ef817267e18 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.549238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a7c71-adba-445c-b340-1ef817267e18 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.549586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a7c71-adba-445c-b340-1ef817267e18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.614609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 798/1610] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:46.616032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac74a2a-bec4-49e7-899b-544e29d7966c None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.616423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac74a2a-bec4-49e7-899b-544e29d7966c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.616616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a7c71-adba-445c-b340-1ef817267e18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:46.616954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac74a2a-bec4-49e7-899b-544e29d7966c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.618843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a7c71-adba-445c-b340-1ef817267e18 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:46.625165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:46.631091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 813/1611] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:46.632166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0686627-f129-4ce5-b0da-672af8fd72f1 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.632532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0686627-f129-4ce5-b0da-672af8fd72f1 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.632824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0686627-f129-4ce5-b0da-672af8fd72f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.649218 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-9ac74a2a-bec4-49e7-899b-544e29d7966c None None] MFA Rules not processed for user `19d048358434468c8e48264c10f9e033`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:46.650938 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9ac74a2a-bec4-49e7-899b-544e29d7966c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:46.662509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-e0686627-f129-4ce5-b0da-672af8fd72f1 None None] MFA Rules not processed for user `fd15e9fec2af4261990e3b6bcb5dea8c`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:46.664109 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e0686627-f129-4ce5-b0da-672af8fd72f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:46.690813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:46.725140 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 799/1612] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:46.726583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c10959-8417-40e4-b57b-8d6033ef28c8 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.726970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c10959-8417-40e4-b57b-8d6033ef28c8 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.727346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c10959-8417-40e4-b57b-8d6033ef28c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.742139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 814/1613] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:46.744841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b25a2-f30b-46be-b867-ff76677a3530 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.744841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b25a2-f30b-46be-b867-ff76677a3530 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.744841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b25a2-f30b-46be-b867-ff76677a3530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.749771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-16c10959-8417-40e4-b57b-8d6033ef28c8 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:46.751106 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-16c10959-8417-40e4-b57b-8d6033ef28c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:46.759887 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 800/1614] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:46.763733 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c10959-8417-40e4-b57b-8d6033ef28c8 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:46.765946 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-013b25a2-f30b-46be-b867-ff76677a3530 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:46.767319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-013b25a2-f30b-46be-b867-ff76677a3530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:46.775531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 815/1615] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:46.776388 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b25a2-f30b-46be-b867-ff76677a3530 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:46.832776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c10959-8417-40e4-b57b-8d6033ef28c8 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:46.836572 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eaf124-3a20-4c66-a7bd-1f1e33f53243 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:46.837391 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eaf124-3a20-4c66-a7bd-1f1e33f53243 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.837773 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eaf124-3a20-4c66-a7bd-1f1e33f53243 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.838153 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eaf124-3a20-4c66-a7bd-1f1e33f53243 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.848711 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eaf124-3a20-4c66-a7bd-1f1e33f53243 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:46.849027 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b25a2-f30b-46be-b867-ff76677a3530 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:46.851446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eaf124-3a20-4c66-a7bd-1f1e33f53243 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:46.852983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359eea2-d7f8-4f16-96fa-2ffc1218b6c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:46.854694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359eea2-d7f8-4f16-96fa-2ffc1218b6c3 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.854694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359eea2-d7f8-4f16-96fa-2ffc1218b6c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.854694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359eea2-d7f8-4f16-96fa-2ffc1218b6c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.863914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 801/1616] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:46.864277 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359eea2-d7f8-4f16-96fa-2ffc1218b6c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:46.864744 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eaf124-3a20-4c66-a7bd-1f1e33f53243 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:46.866194 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359eea2-d7f8-4f16-96fa-2ffc1218b6c3 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:46.877563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 816/1617] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:46.878567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359eea2-d7f8-4f16-96fa-2ffc1218b6c3 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:46.887836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eaf124-3a20-4c66-a7bd-1f1e33f53243 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:46.891356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359eea2-d7f8-4f16-96fa-2ffc1218b6c3 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:46.898783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13146be7-fa41-4fae-b1d1-206ab6f41dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:46.899338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13146be7-fa41-4fae-b1d1-206ab6f41dfb admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.901120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13146be7-fa41-4fae-b1d1-206ab6f41dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.901120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13146be7-fa41-4fae-b1d1-206ab6f41dfb admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.901753 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b57a54-32ad-430f-b17c-7840051511d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:46.902367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b57a54-32ad-430f-b17c-7840051511d8 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:46.903205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13146be7-fa41-4fae-b1d1-206ab6f41dfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:46.904564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b57a54-32ad-430f-b17c-7840051511d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:46.905037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b57a54-32ad-430f-b17c-7840051511d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:46.905498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b57a54-32ad-430f-b17c-7840051511d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:46.907433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b57a54-32ad-430f-b17c-7840051511d8 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:46.910138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13146be7-fa41-4fae-b1d1-206ab6f41dfb admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:47.000990 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 802/1618] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:46 2022] POST /identity/v3/projects => generated 363 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:47.002514 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13146be7-fa41-4fae-b1d1-206ab6f41dfb admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:47.016602 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 817/1619] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:46 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:47.022788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b57a54-32ad-430f-b17c-7840051511d8 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:47.105801 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13146be7-fa41-4fae-b1d1-206ab6f41dfb admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:47.124188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b57a54-32ad-430f-b17c-7840051511d8 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:47.129110 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d5a0f-a726-4f26-ae86-8c0d94118c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:47.129992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d5a0f-a726-4f26-ae86-8c0d94118c38 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:47.130373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d5a0f-a726-4f26-ae86-8c0d94118c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:47.131852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d5a0f-a726-4f26-ae86-8c0d94118c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:47.154933 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:47.202700 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec2cd2-6503-413f-b172-e76e6a586c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:47.203321 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec2cd2-6503-413f-b172-e76e6a586c3f service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:47.203639 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec2cd2-6503-413f-b172-e76e6a586c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:47.204508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec2cd2-6503-413f-b172-e76e6a586c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:47.221664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:47.246648 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d5a0f-a726-4f26-ae86-8c0d94118c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:47.248989 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d5a0f-a726-4f26-ae86-8c0d94118c38 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:47.265646 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 818/1620] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:47.268332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cd903-9ff5-497d-8745-ae7f1a1188ba None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:47.268723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cd903-9ff5-497d-8745-ae7f1a1188ba None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:47.269026 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cd903-9ff5-497d-8745-ae7f1a1188ba None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:47.270850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 819/1621] 10.222.0.26 () {54 vars in 912 bytes} [Wed Aug 24 07:51:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:51:47.271642 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cd903-9ff5-497d-8745-ae7f1a1188ba None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:47.289594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cd903-9ff5-497d-8745-ae7f1a1188ba None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:47.295375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea95ea1-36b4-4f3a-a953-6c69f0164388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:47.296724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea95ea1-36b4-4f3a-a953-6c69f0164388 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:47.297035 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea95ea1-36b4-4f3a-a953-6c69f0164388 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:47.297973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea95ea1-36b4-4f3a-a953-6c69f0164388 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:47.298351 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea95ea1-36b4-4f3a-a953-6c69f0164388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:47.300734 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec2cd2-6503-413f-b172-e76e6a586c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:47.302363 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea95ea1-36b4-4f3a-a953-6c69f0164388 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:47.303941 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec2cd2-6503-413f-b172-e76e6a586c3f service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:47.317614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 803/1622] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:47.319196 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec2cd2-6503-413f-b172-e76e6a586c3f service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:47.331093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec2cd2-6503-413f-b172-e76e6a586c3f service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:47.337791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1c9ce-c009-47c9-aa76-2a8e1af3d311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:47.338724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1c9ce-c009-47c9-aa76-2a8e1af3d311 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:47.338924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1c9ce-c009-47c9-aa76-2a8e1af3d311 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:47.339183 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1c9ce-c009-47c9-aa76-2a8e1af3d311 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:47.339994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1c9ce-c009-47c9-aa76-2a8e1af3d311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:47.342396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1c9ce-c009-47c9-aa76-2a8e1af3d311 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:47.366257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 820/1623] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:47 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:47.366257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4493c3-0e74-4fe7-aecf-e1c855e733d6 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:47.366257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4493c3-0e74-4fe7-aecf-e1c855e733d6 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:47.366257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4493c3-0e74-4fe7-aecf-e1c855e733d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:47.394943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-2a4493c3-0e74-4fe7-aecf-e1c855e733d6 None None] MFA Rules not processed for user `bdf81f09006a42299c0807f77f16c748`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:47.396616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2a4493c3-0e74-4fe7-aecf-e1c855e733d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:47.454767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 804/1624] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:47 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:47.455967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1c9ce-c009-47c9-aa76-2a8e1af3d311 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:47.484015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1c9ce-c009-47c9-aa76-2a8e1af3d311 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:47.495867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923fda3e-38b3-4ad3-b8ca-75652716f1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:47.496586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fda3e-38b3-4ad3-b8ca-75652716f1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:47.496897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fda3e-38b3-4ad3-b8ca-75652716f1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:47.498418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fda3e-38b3-4ad3-b8ca-75652716f1c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:47.499112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923fda3e-38b3-4ad3-b8ca-75652716f1c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:47.502644 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923fda3e-38b3-4ad3-b8ca-75652716f1c8 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:47.525517 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 821/1625] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 24 07:51:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:47.526497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4493c3-0e74-4fe7-aecf-e1c855e733d6 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:47.534004 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 805/1626] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:47.546967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923fda3e-38b3-4ad3-b8ca-75652716f1c8 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:47.549590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4493c3-0e74-4fe7-aecf-e1c855e733d6 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:47.556001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab1d7b-15d9-453b-940d-2659c17bbe14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:47.556001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab1d7b-15d9-453b-940d-2659c17bbe14 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:47.556272 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab1d7b-15d9-453b-940d-2659c17bbe14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:47.556420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab1d7b-15d9-453b-940d-2659c17bbe14 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:47.557431 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab1d7b-15d9-453b-940d-2659c17bbe14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:47.559337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab1d7b-15d9-453b-940d-2659c17bbe14 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:47.571020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 822/1627] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:47.571934 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab1d7b-15d9-453b-940d-2659c17bbe14 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:47.588805 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab1d7b-15d9-453b-940d-2659c17bbe14 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:47.601343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1eb089-ca2d-4bd4-b6ae-3a7af351e1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:47.602034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1eb089-ca2d-4bd4-b6ae-3a7af351e1ac admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:47.602792 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1eb089-ca2d-4bd4-b6ae-3a7af351e1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:47.603075 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1eb089-ca2d-4bd4-b6ae-3a7af351e1ac admin admin] PATH_INFO: `/v3/projects/ec3c6f87e57a4523a3c45121c5df55e8/users/8a028ff249444ab285c694723465ac72/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:47.633505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1eb089-ca2d-4bd4-b6ae-3a7af351e1ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec3c6f87e57a4523a3c45121c5df55e8, user_id=8a028ff249444ab285c694723465ac72, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:47.636161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1eb089-ca2d-4bd4-b6ae-3a7af351e1ac admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:47.653653 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923fda3e-38b3-4ad3-b8ca-75652716f1c8 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:47.697273 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f02b35-65ee-412b-a90d-937dcef631c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:47.698038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f02b35-65ee-412b-a90d-937dcef631c0 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:47.698326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f02b35-65ee-412b-a90d-937dcef631c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:47.698598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f02b35-65ee-412b-a90d-937dcef631c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:47.723963 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f02b35-65ee-412b-a90d-937dcef631c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:47.724289 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 823/1628] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:47 2022] PUT /identity/v3/projects/ec3c6f87e57a4523a3c45121c5df55e8/users/8a028ff249444ab285c694723465ac72/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:47.725474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f02b35-65ee-412b-a90d-937dcef631c0 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:47.726437 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1eb089-ca2d-4bd4-b6ae-3a7af351e1ac admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:47.750882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 806/1629] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:47.752351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f02b35-65ee-412b-a90d-937dcef631c0 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:47.771107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f02b35-65ee-412b-a90d-937dcef631c0 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:47.860670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:47.881487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1eb089-ca2d-4bd4-b6ae-3a7af351e1ac admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:47.890991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050ad57-a002-4d0c-93dd-eec606ed1e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:47.891811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050ad57-a002-4d0c-93dd-eec606ed1e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:47.892191 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050ad57-a002-4d0c-93dd-eec606ed1e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:47.899100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050ad57-a002-4d0c-93dd-eec606ed1e7e admin admin] PATH_INFO: `/v3/projects/8d17b48ebb7c43548675a6f40a6c0ac6/users/663e632829ca46cf8bbeb70e2b05d1bb/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:47.924300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b534833-792a-4226-b952-d90a639f4078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:47.924791 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050ad57-a002-4d0c-93dd-eec606ed1e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d17b48ebb7c43548675a6f40a6c0ac6, user_id=663e632829ca46cf8bbeb70e2b05d1bb, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:47.925655 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b534833-792a-4226-b952-d90a639f4078 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:47.926230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b534833-792a-4226-b952-d90a639f4078 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:47.926671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b534833-792a-4226-b952-d90a639f4078 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:47.927495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050ad57-a002-4d0c-93dd-eec606ed1e7e admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:47.928862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b534833-792a-4226-b952-d90a639f4078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:47.931767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b534833-792a-4226-b952-d90a639f4078 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:47.973035 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 807/1630] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:47.974116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b534833-792a-4226-b952-d90a639f4078 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:47.986445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b534833-792a-4226-b952-d90a639f4078 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:48.018494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74a3d3-9f2c-479c-8885-5e6e79931da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:48.019632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a3d3-9f2c-479c-8885-5e6e79931da7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:48.020114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a3d3-9f2c-479c-8885-5e6e79931da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:48.020585 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a3d3-9f2c-479c-8885-5e6e79931da7 admin admin] PATH_INFO: `/v3/projects/ec3c6f87e57a4523a3c45121c5df55e8/users/8a028ff249444ab285c694723465ac72/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:48.023716 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74a3d3-9f2c-479c-8885-5e6e79931da7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec3c6f87e57a4523a3c45121c5df55e8, user_id=8a028ff249444ab285c694723465ac72, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:48.028359 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74a3d3-9f2c-479c-8885-5e6e79931da7 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:48.060793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 824/1632] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:47 2022] PUT /identity/v3/projects/8d17b48ebb7c43548675a6f40a6c0ac6/users/663e632829ca46cf8bbeb70e2b05d1bb/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:48.064510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050ad57-a002-4d0c-93dd-eec606ed1e7e admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:48.069180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 808/1632] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:47 2022] PUT /identity/v3/projects/ec3c6f87e57a4523a3c45121c5df55e8/users/8a028ff249444ab285c694723465ac72/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:48.072021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30e593-628f-4101-b86f-22348a709b94 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:48.074166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30e593-628f-4101-b86f-22348a709b94 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:48.074166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30e593-628f-4101-b86f-22348a709b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:48.104516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:48.134790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-9f30e593-628f-4101-b86f-22348a709b94 None None] MFA Rules not processed for user `8a028ff249444ab285c694723465ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:48.137936 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9f30e593-628f-4101-b86f-22348a709b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:48.247522 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050ad57-a002-4d0c-93dd-eec606ed1e7e admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:48.259036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 809/1633] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:48.261015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edceaeb-f5aa-46ad-827f-1c7b1c9fabe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:48.261834 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edceaeb-f5aa-46ad-827f-1c7b1c9fabe7 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:48.262282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edceaeb-f5aa-46ad-827f-1c7b1c9fabe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:48.262698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edceaeb-f5aa-46ad-827f-1c7b1c9fabe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:48.270823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edceaeb-f5aa-46ad-827f-1c7b1c9fabe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:48.271455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30e593-628f-4101-b86f-22348a709b94 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:48.273067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edceaeb-f5aa-46ad-827f-1c7b1c9fabe7 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:48.287671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 825/1634] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:48.292451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edceaeb-f5aa-46ad-827f-1c7b1c9fabe7 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:48.339933 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edceaeb-f5aa-46ad-827f-1c7b1c9fabe7 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:48.353389 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3db7e9-b23d-42b6-b559-9dd1f1c8ba22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:48.355752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3db7e9-b23d-42b6-b559-9dd1f1c8ba22 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:48.355752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3db7e9-b23d-42b6-b559-9dd1f1c8ba22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:48.355752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3db7e9-b23d-42b6-b559-9dd1f1c8ba22 admin admin] PATH_INFO: `/v3/projects/8d17b48ebb7c43548675a6f40a6c0ac6/users/663e632829ca46cf8bbeb70e2b05d1bb/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:48.359154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30e593-628f-4101-b86f-22348a709b94 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:48.361247 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3db7e9-b23d-42b6-b559-9dd1f1c8ba22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d17b48ebb7c43548675a6f40a6c0ac6, user_id=663e632829ca46cf8bbeb70e2b05d1bb, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:48.363412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26afde0-95b3-4125-81c8-f4efa2c71b3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:48.363926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3db7e9-b23d-42b6-b559-9dd1f1c8ba22 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:48.364487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26afde0-95b3-4125-81c8-f4efa2c71b3e service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:48.365018 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26afde0-95b3-4125-81c8-f4efa2c71b3e service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:48.365414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26afde0-95b3-4125-81c8-f4efa2c71b3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:48.380394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26afde0-95b3-4125-81c8-f4efa2c71b3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:48.382503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26afde0-95b3-4125-81c8-f4efa2c71b3e service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:48.397220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 826/1635] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:48 2022] PUT /identity/v3/projects/8d17b48ebb7c43548675a6f40a6c0ac6/users/663e632829ca46cf8bbeb70e2b05d1bb/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:48.407889 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 810/1636] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:51:48.408197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bbd6e-2489-444a-a117-f99a22c863c6 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:48.408497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bbd6e-2489-444a-a117-f99a22c863c6 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:48.408882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bbd6e-2489-444a-a117-f99a22c863c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:48.461668 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-8c7bbd6e-2489-444a-a117-f99a22c863c6 None None] MFA Rules not processed for user `663e632829ca46cf8bbeb70e2b05d1bb`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:48.463046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8c7bbd6e-2489-444a-a117-f99a22c863c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:48.678790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 827/1637] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:51:48.734897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26afde0-95b3-4125-81c8-f4efa2c71b3e service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:48.745916 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26afde0-95b3-4125-81c8-f4efa2c71b3e service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:48.827200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4f8f3-a587-4473-a228-e44b744a68a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:48.827826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4f8f3-a587-4473-a228-e44b744a68a0 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:48.828899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4f8f3-a587-4473-a228-e44b744a68a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:48.828899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4f8f3-a587-4473-a228-e44b744a68a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:48.842620 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4f8f3-a587-4473-a228-e44b744a68a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:48.846001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4f8f3-a587-4473-a228-e44b744a68a0 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:48.874709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 811/1638] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:48.917911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bbd6e-2489-444a-a117-f99a22c863c6 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:48.929621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bbd6e-2489-444a-a117-f99a22c863c6 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:48.933884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ff6cf-7e92-4b6d-a879-e1eafb0e381f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:48.934736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ff6cf-7e92-4b6d-a879-e1eafb0e381f service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:48.940972 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ff6cf-7e92-4b6d-a879-e1eafb0e381f service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:48.941603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ff6cf-7e92-4b6d-a879-e1eafb0e381f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:49.019563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:49.060840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ff6cf-7e92-4b6d-a879-e1eafb0e381f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:49.069758 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ff6cf-7e92-4b6d-a879-e1eafb0e381f service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:49.083760 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4f8f3-a587-4473-a228-e44b744a68a0 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:49.104768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4f8f3-a587-4473-a228-e44b744a68a0 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:49.325048 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64548e-13bb-4c60-9875-3dcade937b7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:49.325048 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64548e-13bb-4c60-9875-3dcade937b7c service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:49.325048 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64548e-13bb-4c60-9875-3dcade937b7c service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:49.327188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 828/1639] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:49.327536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64548e-13bb-4c60-9875-3dcade937b7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:49.327536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:49.373129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64548e-13bb-4c60-9875-3dcade937b7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:49.375352 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64548e-13bb-4c60-9875-3dcade937b7c service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:49.400167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba3049-5721-4a5d-b97d-704edee73ca0 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:49.400842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba3049-5721-4a5d-b97d-704edee73ca0 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:49.400842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba3049-5721-4a5d-b97d-704edee73ca0 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:49.403721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 829/1640] 10.222.0.26 () {54 vars in 962 bytes} [Wed Aug 24 07:51:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:51:49.404686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 812/1641] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:51:49.427210 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ad4ca-e6c9-466c-bfc3-ecea3fba3f31 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:49.427522 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ad4ca-e6c9-466c-bfc3-ecea3fba3f31 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:49.427766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ad4ca-e6c9-466c-bfc3-ecea3fba3f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:49.469891 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-625ad4ca-e6c9-466c-bfc3-ecea3fba3f31 None None] MFA Rules not processed for user `bdf81f09006a42299c0807f77f16c748`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:49.471140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-625ad4ca-e6c9-466c-bfc3-ecea3fba3f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:49.585749 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 830/1642] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Aug 24 07:51:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:49.589466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cc518-47ee-40f2-b470-e3395d15edb3 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:49.589759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cc518-47ee-40f2-b470-e3395d15edb3 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:49.590508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cc518-47ee-40f2-b470-e3395d15edb3 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:49.596979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 813/1643] 10.222.0.26 () {54 vars in 912 bytes} [Wed Aug 24 07:51:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:51:49.601699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914311c-8d11-47f9-8309-41aeaeb6e7c6 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:49.602293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914311c-8d11-47f9-8309-41aeaeb6e7c6 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:49.602809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914311c-8d11-47f9-8309-41aeaeb6e7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:49.656331 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-7914311c-8d11-47f9-8309-41aeaeb6e7c6 None None] MFA Rules not processed for user `bdf81f09006a42299c0807f77f16c748`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:49.657512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7914311c-8d11-47f9-8309-41aeaeb6e7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:49.668167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 831/1644] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 24 07:51:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:49.917725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cc518-47ee-40f2-b470-e3395d15edb3 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:49.938717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cc518-47ee-40f2-b470-e3395d15edb3 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:49.945224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe7c60-0c78-4395-aa4e-03577884818b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:49.945923 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe7c60-0c78-4395-aa4e-03577884818b service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:49.946274 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe7c60-0c78-4395-aa4e-03577884818b service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:49.946666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe7c60-0c78-4395-aa4e-03577884818b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:49.967687 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe7c60-0c78-4395-aa4e-03577884818b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:49.969641 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe7c60-0c78-4395-aa4e-03577884818b service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:50.026963 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 814/1645] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:52.262155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914311c-8d11-47f9-8309-41aeaeb6e7c6 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:52.459465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914311c-8d11-47f9-8309-41aeaeb6e7c6 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:52.479702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b615d7-c3c0-4c34-bdac-a86036e3494e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:52.489150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b615d7-c3c0-4c34-bdac-a86036e3494e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:52.489849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b615d7-c3c0-4c34-bdac-a86036e3494e admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:52.490517 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b615d7-c3c0-4c34-bdac-a86036e3494e admin admin] PATH_INFO: `/v3/users/39878b432b9c42a9a0e70f72cb010d75` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:52.495579 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b615d7-c3c0-4c34-bdac-a86036e3494e admin admin] RBAC: Authorizing `identity:delete_user(user_id=39878b432b9c42a9a0e70f72cb010d75)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:52.501231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b615d7-c3c0-4c34-bdac-a86036e3494e admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:52.697431 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-23b615d7-c3c0-4c34-bdac-a86036e3494e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39878b432b9c42a9a0e70f72cb010d75'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:52.715321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-23b615d7-c3c0-4c34-bdac-a86036e3494e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39878b432b9c42a9a0e70f72cb010d75'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:52.805951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-23b615d7-c3c0-4c34-bdac-a86036e3494e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39878b432b9c42a9a0e70f72cb010d75'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:52.806844 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-23b615d7-c3c0-4c34-bdac-a86036e3494e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39878b432b9c42a9a0e70f72cb010d75'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:52.818281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:52.838941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-23b615d7-c3c0-4c34-bdac-a86036e3494e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39878b432b9c42a9a0e70f72cb010d75'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:52.851514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 832/1646] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:51:52 2022] DELETE /identity/v3/users/39878b432b9c42a9a0e70f72cb010d75 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:52.856538 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea73d48-200f-4c76-a4c9-5210c8690e56 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:52.857642 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea73d48-200f-4c76-a4c9-5210c8690e56 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:52.857757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea73d48-200f-4c76-a4c9-5210c8690e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:52.913157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-aea73d48-200f-4c76-a4c9-5210c8690e56 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:52.914396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-aea73d48-200f-4c76-a4c9-5210c8690e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:53.025090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 815/1647] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:53.061322 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b615d7-c3c0-4c34-bdac-a86036e3494e admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:53.063455 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-23b615d7-c3c0-4c34-bdac-a86036e3494e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:53.207575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b615d7-c3c0-4c34-bdac-a86036e3494e admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:53.216862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae21a8-2a81-477a-973e-b730efbaa737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:53.217878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae21a8-2a81-477a-973e-b730efbaa737 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:53.218373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae21a8-2a81-477a-973e-b730efbaa737 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:53.218811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae21a8-2a81-477a-973e-b730efbaa737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:53.246283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae21a8-2a81-477a-973e-b730efbaa737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:53.248859 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae21a8-2a81-477a-973e-b730efbaa737 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:53.264258 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 833/1648] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:54.408252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e7887-0b37-47d3-a849-d33aa40a908e None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:54.408772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e7887-0b37-47d3-a849-d33aa40a908e None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:54.408850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e7887-0b37-47d3-a849-d33aa40a908e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:54.474848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-bc9e7887-0b37-47d3-a849-d33aa40a908e None None] MFA Rules not processed for user `aee01beeba5b48c3b6c25489743de40e`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:54.476105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-bc9e7887-0b37-47d3-a849-d33aa40a908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:54.594705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 816/1649] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:54.609524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae21a8-2a81-477a-973e-b730efbaa737 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:54.611416 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d2ae21a8-2a81-477a-973e-b730efbaa737 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:54.702705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae21a8-2a81-477a-973e-b730efbaa737 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:54.716986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec19b03-d2c6-4bf0-b71f-f1ce060632ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:54.717829 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec19b03-d2c6-4bf0-b71f-f1ce060632ad service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:54.718302 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec19b03-d2c6-4bf0-b71f-f1ce060632ad service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:54.719555 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e7887-0b37-47d3-a849-d33aa40a908e None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:54.721238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-bc9e7887-0b37-47d3-a849-d33aa40a908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:54.733050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec19b03-d2c6-4bf0-b71f-f1ce060632ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:54.745739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec19b03-d2c6-4bf0-b71f-f1ce060632ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:54.748984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec19b03-d2c6-4bf0-b71f-f1ce060632ad service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:54.775673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 834/1650] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:54.781299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e7887-0b37-47d3-a849-d33aa40a908e None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:54.793730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:54.797119 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:54.797979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:54.819414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3 admin admin] PATH_INFO: `/v3/projects/8e0daa1f56f4432d880f2eb4b85391fb` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:54.819414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e0daa1f56f4432d880f2eb4b85391fb)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:54.819414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:55.034205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e0daa1f56f4432d880f2eb4b85391fb', 'request_id': 'req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:55.037263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:55.042219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e0daa1f56f4432d880f2eb4b85391fb', 'request_id': 'req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:55.133128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e0daa1f56f4432d880f2eb4b85391fb', 'request_id': 'req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:55.187351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3 admin admin] The token cache is being invalidate because project 8e0daa1f56f4432d880f2eb4b85391fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:51:55.188166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:55.189251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-efc56a92-d4e6-4c94-9ff3-bdf34c7f06d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:55.192153 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 817/1651] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:51:54 2022] DELETE /identity/v3/projects/8e0daa1f56f4432d880f2eb4b85391fb => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:51:55.197240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91d43b-6e15-4e84-9656-d0202b2d0fdb None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:55.197556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91d43b-6e15-4e84-9656-d0202b2d0fdb None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:55.197823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91d43b-6e15-4e84-9656-d0202b2d0fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:55.279234 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-3c91d43b-6e15-4e84-9656-d0202b2d0fdb None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:55.280333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3c91d43b-6e15-4e84-9656-d0202b2d0fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:55.468616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 835/1652] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:55.476178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f69a01-a0fc-44ac-9227-c3b6df1103de None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:55.476673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f69a01-a0fc-44ac-9227-c3b6df1103de None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:55.476760 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f69a01-a0fc-44ac-9227-c3b6df1103de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:55.521770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d1f69a01-a0fc-44ac-9227-c3b6df1103de None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:55.523221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d1f69a01-a0fc-44ac-9227-c3b6df1103de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:55.538759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 818/1653] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:55.545785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91d43b-6e15-4e84-9656-d0202b2d0fdb None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:55.568240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91d43b-6e15-4e84-9656-d0202b2d0fdb None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:55.605500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f5a37-d5bc-4b0d-8506-cb9ac2d3393a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:55.610629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f5a37-d5bc-4b0d-8506-cb9ac2d3393a admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:55.611276 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f5a37-d5bc-4b0d-8506-cb9ac2d3393a admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:55.611556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f5a37-d5bc-4b0d-8506-cb9ac2d3393a admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:55.613284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f5a37-d5bc-4b0d-8506-cb9ac2d3393a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:51:55.615248 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f5a37-d5bc-4b0d-8506-cb9ac2d3393a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:55.618570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f5a37-d5bc-4b0d-8506-cb9ac2d3393a admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:55.656945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 836/1654] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:51:55.661445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f69a01-a0fc-44ac-9227-c3b6df1103de None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:55.673125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f69a01-a0fc-44ac-9227-c3b6df1103de None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:55.687936 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69f6b9-ca68-43b9-812f-06d936493fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:55.688906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69f6b9-ca68-43b9-812f-06d936493fbf admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:55.690080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69f6b9-ca68-43b9-812f-06d936493fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:55.693149 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69f6b9-ca68-43b9-812f-06d936493fbf admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:55.694063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69f6b9-ca68-43b9-812f-06d936493fbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:55.698621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69f6b9-ca68-43b9-812f-06d936493fbf admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:55.897413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 819/1655] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:55 2022] POST /identity/v3/projects => generated 373 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:55.901404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f5a37-d5bc-4b0d-8506-cb9ac2d3393a admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:55.921893 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f5a37-d5bc-4b0d-8506-cb9ac2d3393a admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:56.079379 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3f305-cb0e-4fb6-8fc0-985a5e1474b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:56.080268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3f305-cb0e-4fb6-8fc0-985a5e1474b8 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:56.080556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3f305-cb0e-4fb6-8fc0-985a5e1474b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:56.080864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3f305-cb0e-4fb6-8fc0-985a5e1474b8 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:56.081510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3f305-cb0e-4fb6-8fc0-985a5e1474b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:56.083497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3f305-cb0e-4fb6-8fc0-985a5e1474b8 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:56.125892 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:56.202856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 837/1656] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:55 2022] POST /identity/v3/users => generated 287 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:56.208676 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69f6b9-ca68-43b9-812f-06d936493fbf admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:56.241028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69f6b9-ca68-43b9-812f-06d936493fbf admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:56.257491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4ec28-8930-4a06-b750-6d35c96a8f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:56.258157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4ec28-8930-4a06-b750-6d35c96a8f48 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:56.258345 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4ec28-8930-4a06-b750-6d35c96a8f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:56.258623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4ec28-8930-4a06-b750-6d35c96a8f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:56.263747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4ec28-8930-4a06-b750-6d35c96a8f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:56.265500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4ec28-8930-4a06-b750-6d35c96a8f48 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:56.295633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 820/1657] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:56.298743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3f305-cb0e-4fb6-8fc0-985a5e1474b8 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:56.310129 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3f305-cb0e-4fb6-8fc0-985a5e1474b8 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:56.323226 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396d54f-8411-48ee-bcb0-e4b2b2e37ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:56.324408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396d54f-8411-48ee-bcb0-e4b2b2e37ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:56.324980 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396d54f-8411-48ee-bcb0-e4b2b2e37ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:56.326150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396d54f-8411-48ee-bcb0-e4b2b2e37ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:56.327024 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396d54f-8411-48ee-bcb0-e4b2b2e37ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:56.328926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396d54f-8411-48ee-bcb0-e4b2b2e37ac9 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:56.353852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4ec28-8930-4a06-b750-6d35c96a8f48 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:56.361673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 838/1658] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:56.368598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4ec28-8930-4a06-b750-6d35c96a8f48 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:56.384222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b469c-b02b-4c33-aa78-99cb5e3b4d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:56.384222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b469c-b02b-4c33-aa78-99cb5e3b4d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:56.384222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b469c-b02b-4c33-aa78-99cb5e3b4d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:56.384222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b469c-b02b-4c33-aa78-99cb5e3b4d04 admin admin] PATH_INFO: `/v3/projects/5bddfa6db5d445c484e42bc05a87eeec/users/001aa7393faa46f191a0e2fbb30e085f/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:56.413894 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b469c-b02b-4c33-aa78-99cb5e3b4d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bddfa6db5d445c484e42bc05a87eeec, user_id=001aa7393faa46f191a0e2fbb30e085f, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:56.417010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b469c-b02b-4c33-aa78-99cb5e3b4d04 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:56.496203 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 821/1659] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:56 2022] PUT /identity/v3/projects/5bddfa6db5d445c484e42bc05a87eeec/users/001aa7393faa46f191a0e2fbb30e085f/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:51:56.507144 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7afc94-c745-45c6-b266-8794f4447da4 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:56.507927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7afc94-c745-45c6-b266-8794f4447da4 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:56.508405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7afc94-c745-45c6-b266-8794f4447da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:56.580703 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-5a7afc94-c745-45c6-b266-8794f4447da4 None None] MFA Rules not processed for user `001aa7393faa46f191a0e2fbb30e085f`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:56.581928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5a7afc94-c745-45c6-b266-8794f4447da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:56.745955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 839/1660] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:56.752082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b469c-b02b-4c33-aa78-99cb5e3b4d04 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:56.753890 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-787b469c-b02b-4c33-aa78-99cb5e3b4d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:56.959411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b469c-b02b-4c33-aa78-99cb5e3b4d04 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:56.967185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14707b2-2660-4249-a2b6-20222abc7e8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:56.968136 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14707b2-2660-4249-a2b6-20222abc7e8e service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:56.968623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14707b2-2660-4249-a2b6-20222abc7e8e service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:56.969151 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14707b2-2660-4249-a2b6-20222abc7e8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:56.979289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14707b2-2660-4249-a2b6-20222abc7e8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:56.981685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14707b2-2660-4249-a2b6-20222abc7e8e service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:57.009145 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 822/1661] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:57.337726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7afc94-c745-45c6-b266-8794f4447da4 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:57.477599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7afc94-c745-45c6-b266-8794f4447da4 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:57.492401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124d2e1-14a7-4547-b1e4-2d9a785228fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:57.494575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124d2e1-14a7-4547-b1e4-2d9a785228fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:57.494575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124d2e1-14a7-4547-b1e4-2d9a785228fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:57.494575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124d2e1-14a7-4547-b1e4-2d9a785228fc admin admin] PATH_INFO: `/v3/users/001aa7393faa46f191a0e2fbb30e085f` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:57.495576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124d2e1-14a7-4547-b1e4-2d9a785228fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=001aa7393faa46f191a0e2fbb30e085f)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:57.498669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124d2e1-14a7-4547-b1e4-2d9a785228fc admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:57.514661 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:57.744556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-9124d2e1-14a7-4547-b1e4-2d9a785228fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '001aa7393faa46f191a0e2fbb30e085f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:57.757408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-9124d2e1-14a7-4547-b1e4-2d9a785228fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '001aa7393faa46f191a0e2fbb30e085f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:57.845122 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-9124d2e1-14a7-4547-b1e4-2d9a785228fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '001aa7393faa46f191a0e2fbb30e085f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:57.845765 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-9124d2e1-14a7-4547-b1e4-2d9a785228fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '001aa7393faa46f191a0e2fbb30e085f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:57.870949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-9124d2e1-14a7-4547-b1e4-2d9a785228fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '001aa7393faa46f191a0e2fbb30e085f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:51:57.888877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 840/1662] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:51:57 2022] DELETE /identity/v3/users/001aa7393faa46f191a0e2fbb30e085f => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:57.893343 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfc3a7-3c61-40d9-994a-dedd6f9b68c8 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:57.895844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfc3a7-3c61-40d9-994a-dedd6f9b68c8 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:57.895844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfc3a7-3c61-40d9-994a-dedd6f9b68c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:57.922892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-bcbfc3a7-3c61-40d9-994a-dedd6f9b68c8 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:57.930798 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-bcbfc3a7-3c61-40d9-994a-dedd6f9b68c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:58.027699 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:58.051370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 823/1663] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:58.386273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124d2e1-14a7-4547-b1e4-2d9a785228fc admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:58.389119 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9124d2e1-14a7-4547-b1e4-2d9a785228fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:58.526629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124d2e1-14a7-4547-b1e4-2d9a785228fc admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:58.538695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88f104-65d3-41f1-a206-21af3d49339c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:58.542919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88f104-65d3-41f1-a206-21af3d49339c service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:58.544481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88f104-65d3-41f1-a206-21af3d49339c service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:58.545077 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88f104-65d3-41f1-a206-21af3d49339c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:58.561528 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88f104-65d3-41f1-a206-21af3d49339c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:58.565120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88f104-65d3-41f1-a206-21af3d49339c service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:58.585107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 841/1664] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:51:59.206133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafeea4-1bd7-4b5e-8829-402fef8bc630 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:59.206784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafeea4-1bd7-4b5e-8829-402fef8bc630 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:59.206784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafeea4-1bd7-4b5e-8829-402fef8bc630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:59.245649 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-0eafeea4-1bd7-4b5e-8829-402fef8bc630 None None] MFA Rules not processed for user `aee01beeba5b48c3b6c25489743de40e`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:59.247824 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0eafeea4-1bd7-4b5e-8829-402fef8bc630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:59.376303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 824/1665] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:51:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:00.286511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88f104-65d3-41f1-a206-21af3d49339c service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:00.287247 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-dc88f104-65d3-41f1-a206-21af3d49339c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:00.309200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88f104-65d3-41f1-a206-21af3d49339c service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:00.354971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba4569-1354-446f-90a5-936ec9981e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:00.355645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba4569-1354-446f-90a5-936ec9981e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:00.355928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba4569-1354-446f-90a5-936ec9981e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:00.356201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba4569-1354-446f-90a5-936ec9981e06 admin admin] PATH_INFO: `/v3/projects/5bddfa6db5d445c484e42bc05a87eeec` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:00.364421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba4569-1354-446f-90a5-936ec9981e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bddfa6db5d445c484e42bc05a87eeec)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:00.364421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: WARNING py.warnings [None req-e3ba4569-1354-446f-90a5-936ec9981e06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:52:00.364421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: warnings.warn(msg) Aug 24 07:52:00.364421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: Aug 24 07:52:00.364421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba4569-1354-446f-90a5-936ec9981e06 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:00.611268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e3ba4569-1354-446f-90a5-936ec9981e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bddfa6db5d445c484e42bc05a87eeec', 'request_id': 'req-e3ba4569-1354-446f-90a5-936ec9981e06'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:00.644367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e3ba4569-1354-446f-90a5-936ec9981e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bddfa6db5d445c484e42bc05a87eeec', 'request_id': 'req-e3ba4569-1354-446f-90a5-936ec9981e06'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:00.661496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e3ba4569-1354-446f-90a5-936ec9981e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bddfa6db5d445c484e42bc05a87eeec', 'request_id': 'req-e3ba4569-1354-446f-90a5-936ec9981e06'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:00.774313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e3ba4569-1354-446f-90a5-936ec9981e06 admin admin] The token cache is being invalidate because project 5bddfa6db5d445c484e42bc05a87eeec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:00.774313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e3ba4569-1354-446f-90a5-936ec9981e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:00.774313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e3ba4569-1354-446f-90a5-936ec9981e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:00.774313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 842/1666] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:00 2022] DELETE /identity/v3/projects/5bddfa6db5d445c484e42bc05a87eeec => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:00.775424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4b3ff-4790-45a4-9c35-bede6ca260a5 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:00.775898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4b3ff-4790-45a4-9c35-bede6ca260a5 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:00.775987 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4b3ff-4790-45a4-9c35-bede6ca260a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:00.809120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-b7a4b3ff-4790-45a4-9c35-bede6ca260a5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:00.811464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b7a4b3ff-4790-45a4-9c35-bede6ca260a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:00.839857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a50a70-99e8-47b7-89ac-015a44903457 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:00.840158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a50a70-99e8-47b7-89ac-015a44903457 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:00.841215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a50a70-99e8-47b7-89ac-015a44903457 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:00.843049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 843/1667] 10.222.0.26 () {54 vars in 912 bytes} [Wed Aug 24 07:52:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:52:00.857907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d513b5b-7fcb-486d-9dcb-098931e2b4f5 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:00.858397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d513b5b-7fcb-486d-9dcb-098931e2b4f5 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:00.859514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d513b5b-7fcb-486d-9dcb-098931e2b4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:00.994785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:01.012019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 825/1668] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:01.019602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352efd4-3ff3-4ee1-8bf7-c234f94055f2 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:01.021388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352efd4-3ff3-4ee1-8bf7-c234f94055f2 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:01.021830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352efd4-3ff3-4ee1-8bf7-c234f94055f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:01.036176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0d513b5b-7fcb-486d-9dcb-098931e2b4f5 None None] MFA Rules not processed for user `46e61402c01c4f9ba5f18e94960bc313`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:01.044289 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0d513b5b-7fcb-486d-9dcb-098931e2b4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:01.070595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-2352efd4-3ff3-4ee1-8bf7-c234f94055f2 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:01.073716 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2352efd4-3ff3-4ee1-8bf7-c234f94055f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:01.098299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 826/1669] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:01.102907 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352efd4-3ff3-4ee1-8bf7-c234f94055f2 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:01.123940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352efd4-3ff3-4ee1-8bf7-c234f94055f2 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:01.148243 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564389f8-d795-4e09-af01-80e799dd2632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:01.149134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564389f8-d795-4e09-af01-80e799dd2632 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:01.149570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564389f8-d795-4e09-af01-80e799dd2632 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:01.150828 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564389f8-d795-4e09-af01-80e799dd2632 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:01.151626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564389f8-d795-4e09-af01-80e799dd2632 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:01.152068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564389f8-d795-4e09-af01-80e799dd2632 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:01.155586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564389f8-d795-4e09-af01-80e799dd2632 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:01.189585 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 827/1670] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:52:01.194101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564389f8-d795-4e09-af01-80e799dd2632 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:01.214176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 844/1671] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 24 07:52:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:01.227893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564389f8-d795-4e09-af01-80e799dd2632 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:01.235312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35219a68-16f0-44c9-b7d5-ab6dc9f030a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:01.235935 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35219a68-16f0-44c9-b7d5-ab6dc9f030a4 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:01.236268 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35219a68-16f0-44c9-b7d5-ab6dc9f030a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:01.236547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35219a68-16f0-44c9-b7d5-ab6dc9f030a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:01.237690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35219a68-16f0-44c9-b7d5-ab6dc9f030a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:01.239748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35219a68-16f0-44c9-b7d5-ab6dc9f030a4 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:01.350021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 828/1672] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:01 2022] POST /identity/v3/projects => generated 379 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:01.352259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d513b5b-7fcb-486d-9dcb-098931e2b4f5 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:01.365369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d513b5b-7fcb-486d-9dcb-098931e2b4f5 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:01.385093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35219a68-16f0-44c9-b7d5-ab6dc9f030a4 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:01.387040 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-35219a68-16f0-44c9-b7d5-ab6dc9f030a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:01.548421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35219a68-16f0-44c9-b7d5-ab6dc9f030a4 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:01.556585 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c6ce1-6f16-43cb-bfce-0766cf57aa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:01.561427 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6ce1-6f16-43cb-bfce-0766cf57aa42 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:01.561908 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6ce1-6f16-43cb-bfce-0766cf57aa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:01.564332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6ce1-6f16-43cb-bfce-0766cf57aa42 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:01.564332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c6ce1-6f16-43cb-bfce-0766cf57aa42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:01.564523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2884a0d-cbda-44f0-bbd8-d1997ad7a361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:01.564523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2884a0d-cbda-44f0-bbd8-d1997ad7a361 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:01.564523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2884a0d-cbda-44f0-bbd8-d1997ad7a361 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:01.564788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2884a0d-cbda-44f0-bbd8-d1997ad7a361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:01.568526 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c6ce1-6f16-43cb-bfce-0766cf57aa42 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:01.582387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2884a0d-cbda-44f0-bbd8-d1997ad7a361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:01.583885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2884a0d-cbda-44f0-bbd8-d1997ad7a361 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:01.604052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 829/1673] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:01.715348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 845/1674] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:01 2022] POST /identity/v3/users => generated 290 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:01.717747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2884a0d-cbda-44f0-bbd8-d1997ad7a361 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:01.741129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2884a0d-cbda-44f0-bbd8-d1997ad7a361 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:01.752458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d359920-901b-41f9-a01c-d3126c446e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:01.753107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d359920-901b-41f9-a01c-d3126c446e5c admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:01.753448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d359920-901b-41f9-a01c-d3126c446e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:01.753703 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d359920-901b-41f9-a01c-d3126c446e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:01.754542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d359920-901b-41f9-a01c-d3126c446e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:01.756150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d359920-901b-41f9-a01c-d3126c446e5c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:01.759993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:01.803366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c6ce1-6f16-43cb-bfce-0766cf57aa42 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:01.816238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c6ce1-6f16-43cb-bfce-0766cf57aa42 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:01.816952 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 830/1675] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:01.853415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30541d4f-ce24-4229-8512-96a8ac9a6d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:01.854248 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30541d4f-ce24-4229-8512-96a8ac9a6d43 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:01.854547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30541d4f-ce24-4229-8512-96a8ac9a6d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:01.854831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30541d4f-ce24-4229-8512-96a8ac9a6d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:01.855484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30541d4f-ce24-4229-8512-96a8ac9a6d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:01.857532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30541d4f-ce24-4229-8512-96a8ac9a6d43 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:01.912139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 846/1676] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:01.914651 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d359920-901b-41f9-a01c-d3126c446e5c admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:01.946027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d359920-901b-41f9-a01c-d3126c446e5c admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:01.969932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8e775-3398-4263-8b5d-635be462d89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:01.970577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8e775-3398-4263-8b5d-635be462d89d admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:01.970857 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8e775-3398-4263-8b5d-635be462d89d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:01.971131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8e775-3398-4263-8b5d-635be462d89d admin admin] PATH_INFO: `/v3/projects/ed0b02b7f3e14eb699f122372bf5aa25/users/27325b962f1e4009ad6f12d09514a7b2/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:01.995934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8e775-3398-4263-8b5d-635be462d89d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed0b02b7f3e14eb699f122372bf5aa25, user_id=27325b962f1e4009ad6f12d09514a7b2, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:02.007211 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8e775-3398-4263-8b5d-635be462d89d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:02.110586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 831/1677] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:01 2022] PUT /identity/v3/projects/ed0b02b7f3e14eb699f122372bf5aa25/users/27325b962f1e4009ad6f12d09514a7b2/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:02.120980 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9c8d4-367c-4fa7-b19e-d25b72146faa None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:02.121630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9c8d4-367c-4fa7-b19e-d25b72146faa None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:02.122061 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9c8d4-367c-4fa7-b19e-d25b72146faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:02.198390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-d0a9c8d4-367c-4fa7-b19e-d25b72146faa None None] MFA Rules not processed for user `27325b962f1e4009ad6f12d09514a7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:02.199845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d0a9c8d4-367c-4fa7-b19e-d25b72146faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:02.378274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 847/1678] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:02.382180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8e775-3398-4263-8b5d-635be462d89d admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:02.590368 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c882bb-1656-48a0-992b-b97232de53d5 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:02.591483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c882bb-1656-48a0-992b-b97232de53d5 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:02.601193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c882bb-1656-48a0-992b-b97232de53d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:02.615638 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8e775-3398-4263-8b5d-635be462d89d admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:02.638819 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc34bc-fef9-4d0f-80fd-398da6a86936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:02.639669 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc34bc-fef9-4d0f-80fd-398da6a86936 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:02.640205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc34bc-fef9-4d0f-80fd-398da6a86936 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:02.640538 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc34bc-fef9-4d0f-80fd-398da6a86936 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:02.642089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc34bc-fef9-4d0f-80fd-398da6a86936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:02.644247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc34bc-fef9-4d0f-80fd-398da6a86936 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:02.712005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-28c882bb-1656-48a0-992b-b97232de53d5 None None] MFA Rules not processed for user `a610194aaf534d3aabb56c9443b5d56d`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:02.713966 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-28c882bb-1656-48a0-992b-b97232de53d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:02.800020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 832/1679] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:02 2022] POST /identity/v3/projects => generated 381 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:02.805464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc34bc-fef9-4d0f-80fd-398da6a86936 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:02.913233 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc34bc-fef9-4d0f-80fd-398da6a86936 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:02.921181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ea793-873a-4b64-a6c9-24b2e48baeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:02.922248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ea793-873a-4b64-a6c9-24b2e48baeff admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:02.922797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ea793-873a-4b64-a6c9-24b2e48baeff admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:02.924789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ea793-873a-4b64-a6c9-24b2e48baeff admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:02.929486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ea793-873a-4b64-a6c9-24b2e48baeff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:02.932129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ea793-873a-4b64-a6c9-24b2e48baeff admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:02.959243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 848/1680] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:02.960904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e43884-a9e4-4ab3-83f1-c97a6b343759 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:02.961188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e43884-a9e4-4ab3-83f1-c97a6b343759 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:02.961636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e43884-a9e4-4ab3-83f1-c97a6b343759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:03.023983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-e8e43884-a9e4-4ab3-83f1-c97a6b343759 None None] MFA Rules not processed for user `a610194aaf534d3aabb56c9443b5d56d`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:03.023983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e8e43884-a9e4-4ab3-83f1-c97a6b343759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:03.114671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:03.150126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 833/1681] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:02 2022] POST /identity/v3/users => generated 297 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:03.156499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ea793-873a-4b64-a6c9-24b2e48baeff admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:03.180495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 849/1682] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:03.187699 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ea793-873a-4b64-a6c9-24b2e48baeff admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:03.187914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e43884-a9e4-4ab3-83f1-c97a6b343759 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:03.191346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e8e43884-a9e4-4ab3-83f1-c97a6b343759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:03.226388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd02c1-a2f9-41dd-a92c-fa7cf461b011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:03.227182 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd02c1-a2f9-41dd-a92c-fa7cf461b011 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:03.227473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd02c1-a2f9-41dd-a92c-fa7cf461b011 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:03.227746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd02c1-a2f9-41dd-a92c-fa7cf461b011 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:03.230869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd02c1-a2f9-41dd-a92c-fa7cf461b011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:03.233944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd02c1-a2f9-41dd-a92c-fa7cf461b011 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:03.259937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 834/1683] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:03.264249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd02c1-a2f9-41dd-a92c-fa7cf461b011 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:03.285678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd02c1-a2f9-41dd-a92c-fa7cf461b011 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:03.294695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cec8a5-cf22-48ee-9f7d-52458ff31a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:03.298279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cec8a5-cf22-48ee-9f7d-52458ff31a03 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:03.298279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cec8a5-cf22-48ee-9f7d-52458ff31a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:03.298279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cec8a5-cf22-48ee-9f7d-52458ff31a03 admin admin] PATH_INFO: `/v3/projects/694039dac8a044fb846b05f0d6b86bfd/users/98ff073a3a3c4fd6bfe18140536500b1/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:03.359021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cec8a5-cf22-48ee-9f7d-52458ff31a03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=694039dac8a044fb846b05f0d6b86bfd, user_id=98ff073a3a3c4fd6bfe18140536500b1, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:03.373200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cec8a5-cf22-48ee-9f7d-52458ff31a03 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:03.443103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e43884-a9e4-4ab3-83f1-c97a6b343759 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:03.456012 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf4e40-3d6c-465a-92f7-6649eb8d05df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:03.457058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf4e40-3d6c-465a-92f7-6649eb8d05df service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:03.457497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf4e40-3d6c-465a-92f7-6649eb8d05df service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:03.458004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf4e40-3d6c-465a-92f7-6649eb8d05df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:03.470827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 835/1684] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:03 2022] PUT /identity/v3/projects/694039dac8a044fb846b05f0d6b86bfd/users/98ff073a3a3c4fd6bfe18140536500b1/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:03.472125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21d177-2329-4971-980c-2217a0d5f6fd None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:03.472446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21d177-2329-4971-980c-2217a0d5f6fd None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:03.472746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21d177-2329-4971-980c-2217a0d5f6fd None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:03.478436 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 836/1685] 10.222.0.26 () {54 vars in 961 bytes} [Wed Aug 24 07:52:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:52:03.479348 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21d177-2329-4971-980c-2217a0d5f6fd None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:03.484311 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf4e40-3d6c-465a-92f7-6649eb8d05df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:03.488343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf4e40-3d6c-465a-92f7-6649eb8d05df service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:03.508312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 850/1686] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:03.514590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602222a-35cd-4490-b305-ea90c0af8cb4 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:03.514590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602222a-35cd-4490-b305-ea90c0af8cb4 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:03.514590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602222a-35cd-4490-b305-ea90c0af8cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:03.565601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:03.579689 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-b602222a-35cd-4490-b305-ea90c0af8cb4 None None] MFA Rules not processed for user `ea3816d5ae054741ac2c5a2eec3eb617`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:03.588330 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b602222a-35cd-4490-b305-ea90c0af8cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:03.757011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21d177-2329-4971-980c-2217a0d5f6fd None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:03.769910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439203c-4991-44e1-8a94-f4e365de9d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:03.772620 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439203c-4991-44e1-8a94-f4e365de9d61 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:03.773078 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439203c-4991-44e1-8a94-f4e365de9d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:03.774573 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439203c-4991-44e1-8a94-f4e365de9d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:03.775395 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439203c-4991-44e1-8a94-f4e365de9d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:03.779107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439203c-4991-44e1-8a94-f4e365de9d61 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:03.818699 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 837/1687] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:03.819944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439203c-4991-44e1-8a94-f4e365de9d61 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:03.824773 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 851/1688] 10.222.0.26 () {58 vars in 1058 bytes} [Wed Aug 24 07:52:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:03.840468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602222a-35cd-4490-b305-ea90c0af8cb4 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:03.852478 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439203c-4991-44e1-8a94-f4e365de9d61 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:03.871045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969aba14-2e60-4862-b9f2-7191a19f56ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:03.873355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969aba14-2e60-4862-b9f2-7191a19f56ea admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:03.873633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969aba14-2e60-4862-b9f2-7191a19f56ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:03.875440 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969aba14-2e60-4862-b9f2-7191a19f56ea admin admin] PATH_INFO: `/v3/projects/694039dac8a044fb846b05f0d6b86bfd/users/98ff073a3a3c4fd6bfe18140536500b1/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:03.876499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602222a-35cd-4490-b305-ea90c0af8cb4 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:03.880859 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969aba14-2e60-4862-b9f2-7191a19f56ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=694039dac8a044fb846b05f0d6b86bfd, user_id=98ff073a3a3c4fd6bfe18140536500b1, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:03.887355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969aba14-2e60-4862-b9f2-7191a19f56ea admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:03.889853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d7885-99df-4367-9158-609b0a518bc5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:03.890496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d7885-99df-4367-9158-609b0a518bc5 service placement] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:03.890804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d7885-99df-4367-9158-609b0a518bc5 service placement] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:03.893039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d7885-99df-4367-9158-609b0a518bc5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:03.897344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3a9d7885-99df-4367-9158-609b0a518bc5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:03.918729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 838/1689] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:03 2022] PUT /identity/v3/projects/694039dac8a044fb846b05f0d6b86bfd/users/98ff073a3a3c4fd6bfe18140536500b1/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:03.932536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a57949-f07d-40e9-8415-2ab7f80f1628 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:03.933165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a57949-f07d-40e9-8415-2ab7f80f1628 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:03.933390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d7885-99df-4367-9158-609b0a518bc5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:03.934148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a57949-f07d-40e9-8415-2ab7f80f1628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:03.935931 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d7885-99df-4367-9158-609b0a518bc5 service placement] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:03.963476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 852/1690] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:03.990610 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-50a57949-f07d-40e9-8415-2ab7f80f1628 None None] MFA Rules not processed for user `98ff073a3a3c4fd6bfe18140536500b1`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:03.992280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-50a57949-f07d-40e9-8415-2ab7f80f1628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:04.106401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 839/1691] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:04.113638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d7885-99df-4367-9158-609b0a518bc5 service placement] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:04.120501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3a9d7885-99df-4367-9158-609b0a518bc5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:04.253943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d7885-99df-4367-9158-609b0a518bc5 service placement] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:04.334216 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99deed-5399-47bf-a7e1-08ac886a5651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:04.334911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99deed-5399-47bf-a7e1-08ac886a5651 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:04.335201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99deed-5399-47bf-a7e1-08ac886a5651 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:04.335497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99deed-5399-47bf-a7e1-08ac886a5651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:04.349213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99deed-5399-47bf-a7e1-08ac886a5651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:04.351098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99deed-5399-47bf-a7e1-08ac886a5651 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:04.363382 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 853/1692] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:04.559235 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee5500-72b5-47fb-85f8-ff62ecc565b9 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:04.560259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee5500-72b5-47fb-85f8-ff62ecc565b9 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:04.560717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee5500-72b5-47fb-85f8-ff62ecc565b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:04.631489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-b7ee5500-72b5-47fb-85f8-ff62ecc565b9 None None] MFA Rules not processed for user `e23edadcfe3d4a0b81b13e278682ca2f`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:04.633116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b7ee5500-72b5-47fb-85f8-ff62ecc565b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:04.640599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:04.713157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99deed-5399-47bf-a7e1-08ac886a5651 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:04.715336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6c99deed-5399-47bf-a7e1-08ac886a5651 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:04.842720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 840/1693] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:04.846164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee5500-72b5-47fb-85f8-ff62ecc565b9 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:04.859576 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee5500-72b5-47fb-85f8-ff62ecc565b9 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:04.866346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f742da-7fe8-40f9-81a8-8f03070e84cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:04.867426 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f742da-7fe8-40f9-81a8-8f03070e84cf service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:04.867893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f742da-7fe8-40f9-81a8-8f03070e84cf service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:04.868367 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f742da-7fe8-40f9-81a8-8f03070e84cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:04.882864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:04.913164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99deed-5399-47bf-a7e1-08ac886a5651 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:04.938172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cad467-c0d4-4f61-a390-7226b47326a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:04.939301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cad467-c0d4-4f61-a390-7226b47326a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:04.939789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cad467-c0d4-4f61-a390-7226b47326a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:04.940243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cad467-c0d4-4f61-a390-7226b47326a5 admin admin] PATH_INFO: `/v3/users/ee9f4acd6ad247b99ac8a55600c701ff` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:04.945383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cad467-c0d4-4f61-a390-7226b47326a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee9f4acd6ad247b99ac8a55600c701ff)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:04.947639 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cad467-c0d4-4f61-a390-7226b47326a5 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:04.968281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f742da-7fe8-40f9-81a8-8f03070e84cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:04.970563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f742da-7fe8-40f9-81a8-8f03070e84cf service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:04.984829 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 841/1694] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:04.986789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cbdd1-a1e3-4505-965e-62e791677196 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:04.987481 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cbdd1-a1e3-4505-965e-62e791677196 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:04.988051 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cbdd1-a1e3-4505-965e-62e791677196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:05.017988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-876cbdd1-a1e3-4505-965e-62e791677196 None None] MFA Rules not processed for user `e23edadcfe3d4a0b81b13e278682ca2f`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:05.019542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-876cbdd1-a1e3-4505-965e-62e791677196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:05.037434 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 842/1695] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:05.051638 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cbdd1-a1e3-4505-965e-62e791677196 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:05.053511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-876cbdd1-a1e3-4505-965e-62e791677196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:05.179673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cbdd1-a1e3-4505-965e-62e791677196 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:05.186460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13470f-cf01-4b51-b980-ceaf80db0090 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:05.207045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13470f-cf01-4b51-b980-ceaf80db0090 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:05.207235 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13470f-cf01-4b51-b980-ceaf80db0090 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:05.207518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13470f-cf01-4b51-b980-ceaf80db0090 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:05.245825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-d0cad467-c0d4-4f61-a390-7226b47326a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee9f4acd6ad247b99ac8a55600c701ff'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:05.253105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13470f-cf01-4b51-b980-ceaf80db0090 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:05.259669 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13470f-cf01-4b51-b980-ceaf80db0090 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:05.268130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-d0cad467-c0d4-4f61-a390-7226b47326a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee9f4acd6ad247b99ac8a55600c701ff'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:05.290667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 843/1696] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:05.354514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:05.354514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-d0cad467-c0d4-4f61-a390-7226b47326a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee9f4acd6ad247b99ac8a55600c701ff'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:05.354514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-d0cad467-c0d4-4f61-a390-7226b47326a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee9f4acd6ad247b99ac8a55600c701ff'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:05.374195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-d0cad467-c0d4-4f61-a390-7226b47326a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee9f4acd6ad247b99ac8a55600c701ff'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:05.397474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 854/1697] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:04 2022] DELETE /identity/v3/users/ee9f4acd6ad247b99ac8a55600c701ff => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:05.402019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815868de-9ade-48b0-bd64-d38cf1a54e69 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:05.402309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815868de-9ade-48b0-bd64-d38cf1a54e69 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:05.402767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815868de-9ade-48b0-bd64-d38cf1a54e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:05.439203 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-815868de-9ade-48b0-bd64-d38cf1a54e69 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:05.440530 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-815868de-9ade-48b0-bd64-d38cf1a54e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:05.580638 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 844/1698] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:05.598284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cad467-c0d4-4f61-a390-7226b47326a5 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:05.600074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d0cad467-c0d4-4f61-a390-7226b47326a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:05.910801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cad467-c0d4-4f61-a390-7226b47326a5 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:05.933325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55150aa-005d-4758-86be-48ed5c936f59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:05.942055 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55150aa-005d-4758-86be-48ed5c936f59 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:05.942535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55150aa-005d-4758-86be-48ed5c936f59 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:05.942948 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55150aa-005d-4758-86be-48ed5c936f59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:05.996314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55150aa-005d-4758-86be-48ed5c936f59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:06.022513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55150aa-005d-4758-86be-48ed5c936f59 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:06.081376 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 855/1699] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:06.357992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815868de-9ade-48b0-bd64-d38cf1a54e69 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:06.360424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-815868de-9ade-48b0-bd64-d38cf1a54e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:06.390093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815868de-9ade-48b0-bd64-d38cf1a54e69 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:06.406637 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:06.407293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:06.407579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:06.407858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] PATH_INFO: `/v3/users/19d048358434468c8e48264c10f9e033` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:06.410843 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d048358434468c8e48264c10f9e033)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:06.412239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: WARNING py.warnings [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:52:06.412239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: warnings.warn(msg) Aug 24 07:52:06.412239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: Aug 24 07:52:06.416397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:06.535581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bdd93-6624-4d47-8394-782729f50534 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:06.535985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bdd93-6624-4d47-8394-782729f50534 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:06.536361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bdd93-6624-4d47-8394-782729f50534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:06.565833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:06.587771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-6e4bdd93-6624-4d47-8394-782729f50534 None None] MFA Rules not processed for user `2ec3daa674814619976aead75459e98c`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:06.589441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6e4bdd93-6624-4d47-8394-782729f50534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:06.844201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 856/1700] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:06.846014 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd324c72-0d45-4e37-8bd4-4a6b7d4f7926 None None] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:06.846588 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd324c72-0d45-4e37-8bd4-4a6b7d4f7926 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:06.846998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd324c72-0d45-4e37-8bd4-4a6b7d4f7926 None None] PATH_INFO: `/` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:06.849122 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 857/1701] 10.222.0.26 () {54 vars in 912 bytes} [Wed Aug 24 07:52:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:52:06.864768 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fe32f-381b-4ac9-9d3d-1aa744c536a2 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:06.865218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fe32f-381b-4ac9-9d3d-1aa744c536a2 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:06.865681 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fe32f-381b-4ac9-9d3d-1aa744c536a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:06.881917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d048358434468c8e48264c10f9e033'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:06.903147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-937fe32f-381b-4ac9-9d3d-1aa744c536a2 None None] MFA Rules not processed for user `46e61402c01c4f9ba5f18e94960bc313`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:06.905630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-937fe32f-381b-4ac9-9d3d-1aa744c536a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:06.928374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 858/1702] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 24 07:52:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:06.929697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fe32f-381b-4ac9-9d3d-1aa744c536a2 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:06.931731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-937fe32f-381b-4ac9-9d3d-1aa744c536a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:06.980091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:07.005538 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d048358434468c8e48264c10f9e033'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:07.006269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d048358434468c8e48264c10f9e033'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:07.038783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d048358434468c8e48264c10f9e033'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:07.058120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fe32f-381b-4ac9-9d3d-1aa744c536a2 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:07.073985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb1f2a-b573-4186-97d9-bb9ba20ab91e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:07.074897 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb1f2a-b573-4186-97d9-bb9ba20ab91e service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:07.077001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb1f2a-b573-4186-97d9-bb9ba20ab91e service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:07.077506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb1f2a-b573-4186-97d9-bb9ba20ab91e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:07.106936 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb1f2a-b573-4186-97d9-bb9ba20ab91e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:07.107831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d048358434468c8e48264c10f9e033'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:07.115175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb1f2a-b573-4186-97d9-bb9ba20ab91e service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:07.128823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 845/1703] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:06 2022] DELETE /identity/v3/users/19d048358434468c8e48264c10f9e033 => generated 0 bytes in 772 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:07.129860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:07.135662 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:07.149190 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 859/1704] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:07.180667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb1f2a-b573-4186-97d9-bb9ba20ab91e service nova] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:07.203825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6bdb1f2a-b573-4186-97d9-bb9ba20ab91e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:07.263207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb1f2a-b573-4186-97d9-bb9ba20ab91e service nova] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:07.284201 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d0e13-0341-4102-9fa0-d655b819dbad admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:07.290919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff582a4a-bb4c-4489-b3ba-28681fee859e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:07.291757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff582a4a-bb4c-4489-b3ba-28681fee859e service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:07.292192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff582a4a-bb4c-4489-b3ba-28681fee859e service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:07.292706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff582a4a-bb4c-4489-b3ba-28681fee859e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:07.296560 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ff582a4a-bb4c-4489-b3ba-28681fee859e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:07.318735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff582a4a-bb4c-4489-b3ba-28681fee859e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:07.320709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff582a4a-bb4c-4489-b3ba-28681fee859e service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:07.333508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 846/1705] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:07.335386 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367b724-da8f-4eb8-ae63-fad391b60d86 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:07.335890 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367b724-da8f-4eb8-ae63-fad391b60d86 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:07.336361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367b724-da8f-4eb8-ae63-fad391b60d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:07.382025 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-2367b724-da8f-4eb8-ae63-fad391b60d86 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:07.384184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2367b724-da8f-4eb8-ae63-fad391b60d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:07.432853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1da5fe-4576-4515-bbfc-d236f8ac582e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:07.433520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1da5fe-4576-4515-bbfc-d236f8ac582e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:07.433804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1da5fe-4576-4515-bbfc-d236f8ac582e admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:07.434088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1da5fe-4576-4515-bbfc-d236f8ac582e admin admin] PATH_INFO: `/v3/projects/b0202ce802094885ac4e7a24c992b166` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:07.436206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1da5fe-4576-4515-bbfc-d236f8ac582e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0202ce802094885ac4e7a24c992b166)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:07.454557 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1da5fe-4576-4515-bbfc-d236f8ac582e admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:07.506131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 847/1706] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:07.546614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367b724-da8f-4eb8-ae63-fad391b60d86 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:07.548267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2367b724-da8f-4eb8-ae63-fad391b60d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:07.583277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367b724-da8f-4eb8-ae63-fad391b60d86 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:07.610419 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb472e05-2ac0-4592-b4d2-2a917c2102f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:07.611072 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb472e05-2ac0-4592-b4d2-2a917c2102f8 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:07.611591 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb472e05-2ac0-4592-b4d2-2a917c2102f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:07.611876 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb472e05-2ac0-4592-b4d2-2a917c2102f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:07.640289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb472e05-2ac0-4592-b4d2-2a917c2102f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:07.641850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb472e05-2ac0-4592-b4d2-2a917c2102f8 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:07.683493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-6e1da5fe-4576-4515-bbfc-d236f8ac582e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0202ce802094885ac4e7a24c992b166', 'request_id': 'req-6e1da5fe-4576-4515-bbfc-d236f8ac582e'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:07.691516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 848/1707] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:07.698603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-6e1da5fe-4576-4515-bbfc-d236f8ac582e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0202ce802094885ac4e7a24c992b166', 'request_id': 'req-6e1da5fe-4576-4515-bbfc-d236f8ac582e'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:07.715249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-6e1da5fe-4576-4515-bbfc-d236f8ac582e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0202ce802094885ac4e7a24c992b166', 'request_id': 'req-6e1da5fe-4576-4515-bbfc-d236f8ac582e'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:07.919131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-6e1da5fe-4576-4515-bbfc-d236f8ac582e admin admin] The token cache is being invalidate because project b0202ce802094885ac4e7a24c992b166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:07.919741 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-6e1da5fe-4576-4515-bbfc-d236f8ac582e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:07.922075 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-6e1da5fe-4576-4515-bbfc-d236f8ac582e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:07.924802 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 860/1708] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:07 2022] DELETE /identity/v3/projects/b0202ce802094885ac4e7a24c992b166 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:07.930606 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e58fa-b092-41ff-a7cd-8e327f7898e0 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:07.930785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e58fa-b092-41ff-a7cd-8e327f7898e0 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:07.931047 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e58fa-b092-41ff-a7cd-8e327f7898e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:07.993772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-bc3e58fa-b092-41ff-a7cd-8e327f7898e0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:07.995167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-bc3e58fa-b092-41ff-a7cd-8e327f7898e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:08.043387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:08.178649 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 849/1709] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:08.187763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790af25-c92a-4552-ae3e-2a53f0747f53 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:08.188932 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790af25-c92a-4552-ae3e-2a53f0747f53 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:08.189538 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790af25-c92a-4552-ae3e-2a53f0747f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:08.268072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-c790af25-c92a-4552-ae3e-2a53f0747f53 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:08.273209 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c790af25-c92a-4552-ae3e-2a53f0747f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:08.335671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 861/1710] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:08.337102 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e58fa-b092-41ff-a7cd-8e327f7898e0 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:08.355074 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e58fa-b092-41ff-a7cd-8e327f7898e0 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:08.367674 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533b882-aecd-4030-bf96-da399d8a4774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:08.368573 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533b882-aecd-4030-bf96-da399d8a4774 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:08.369038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533b882-aecd-4030-bf96-da399d8a4774 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:08.369460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533b882-aecd-4030-bf96-da399d8a4774 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:08.370261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533b882-aecd-4030-bf96-da399d8a4774 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:08.371002 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533b882-aecd-4030-bf96-da399d8a4774 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:08.381840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533b882-aecd-4030-bf96-da399d8a4774 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:08.401483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 850/1711] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:52:08.406644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790af25-c92a-4552-ae3e-2a53f0747f53 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:08.431344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790af25-c92a-4552-ae3e-2a53f0747f53 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:08.485176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80028356-775e-49c1-a1cb-1df0e1f3efa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:08.490393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80028356-775e-49c1-a1cb-1df0e1f3efa6 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:08.490393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80028356-775e-49c1-a1cb-1df0e1f3efa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:08.490393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80028356-775e-49c1-a1cb-1df0e1f3efa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:08.490393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80028356-775e-49c1-a1cb-1df0e1f3efa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:08.494530 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80028356-775e-49c1-a1cb-1df0e1f3efa6 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:08.683475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7063f5a-4d2e-450a-bb0d-198c93d7cb99 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:08.683872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7063f5a-4d2e-450a-bb0d-198c93d7cb99 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:08.684295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7063f5a-4d2e-450a-bb0d-198c93d7cb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:08.694693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 862/1712] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:08 2022] POST /identity/v3/projects => generated 369 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:08.702763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80028356-775e-49c1-a1cb-1df0e1f3efa6 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:08.735167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-c7063f5a-4d2e-450a-bb0d-198c93d7cb99 None None] MFA Rules not processed for user `e23edadcfe3d4a0b81b13e278682ca2f`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:08.738455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c7063f5a-4d2e-450a-bb0d-198c93d7cb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:08.840041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 851/1713] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:08.912044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:09.009796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80028356-775e-49c1-a1cb-1df0e1f3efa6 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:09.016920 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c473118-c0bf-4c45-ace1-fa04b9946530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:09.017983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c473118-c0bf-4c45-ace1-fa04b9946530 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:09.018476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c473118-c0bf-4c45-ace1-fa04b9946530 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:09.019466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c473118-c0bf-4c45-ace1-fa04b9946530 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:09.020319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c473118-c0bf-4c45-ace1-fa04b9946530 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:09.031684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c473118-c0bf-4c45-ace1-fa04b9946530 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:09.201910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7063f5a-4d2e-450a-bb0d-198c93d7cb99 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:09.202044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 863/1714] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:08 2022] POST /identity/v3/users => generated 285 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:09.233102 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7063f5a-4d2e-450a-bb0d-198c93d7cb99 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:09.267397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712164d6-7fee-4132-b3c5-40314227491c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:09.268099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712164d6-7fee-4132-b3c5-40314227491c admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:09.270232 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712164d6-7fee-4132-b3c5-40314227491c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:09.278172 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712164d6-7fee-4132-b3c5-40314227491c admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:09.279493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712164d6-7fee-4132-b3c5-40314227491c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:09.286652 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712164d6-7fee-4132-b3c5-40314227491c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:09.325416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 852/1715] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:09.327016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c473118-c0bf-4c45-ace1-fa04b9946530 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:09.366503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c473118-c0bf-4c45-ace1-fa04b9946530 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:09.384956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80048be-e3a7-43ea-9e28-fd1cd10c38fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:09.388939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80048be-e3a7-43ea-9e28-fd1cd10c38fb admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:09.389506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80048be-e3a7-43ea-9e28-fd1cd10c38fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:09.390190 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80048be-e3a7-43ea-9e28-fd1cd10c38fb admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:09.392760 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80048be-e3a7-43ea-9e28-fd1cd10c38fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:09.398195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80048be-e3a7-43ea-9e28-fd1cd10c38fb admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:09.436062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 864/1716] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:52:09.440695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712164d6-7fee-4132-b3c5-40314227491c admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:09.481689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712164d6-7fee-4132-b3c5-40314227491c admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:09.503198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16c4b2-6b99-4560-af53-dcb621e101a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:09.510371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16c4b2-6b99-4560-af53-dcb621e101a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:09.510371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16c4b2-6b99-4560-af53-dcb621e101a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:09.510371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16c4b2-6b99-4560-af53-dcb621e101a1 admin admin] PATH_INFO: `/v3/projects/b7c01c0b053f4087985778f55bc0345f/users/5fe090e27b2c4669b92a73dd364deb2c/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:09.560221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16c4b2-6b99-4560-af53-dcb621e101a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7c01c0b053f4087985778f55bc0345f, user_id=5fe090e27b2c4669b92a73dd364deb2c, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:09.567016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16c4b2-6b99-4560-af53-dcb621e101a1 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:09.659695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80048be-e3a7-43ea-9e28-fd1cd10c38fb admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:09.661761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e80048be-e3a7-43ea-9e28-fd1cd10c38fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:09.695246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 853/1717] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:09 2022] PUT /identity/v3/projects/b7c01c0b053f4087985778f55bc0345f/users/5fe090e27b2c4669b92a73dd364deb2c/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:09.710898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c806406-4ea0-46a3-9293-a81a9073aef8 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:09.713908 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c806406-4ea0-46a3-9293-a81a9073aef8 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:09.714643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c806406-4ea0-46a3-9293-a81a9073aef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:09.727528 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80048be-e3a7-43ea-9e28-fd1cd10c38fb admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:09.811117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-9c806406-4ea0-46a3-9293-a81a9073aef8 None None] MFA Rules not processed for user `5fe090e27b2c4669b92a73dd364deb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:09.819673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9c806406-4ea0-46a3-9293-a81a9073aef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:09.963743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:09.966747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:09.968792 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:09.970069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] PATH_INFO: `/v3/projects/5a81a64cb57941548f25273bca00ec25` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:09.974450 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a81a64cb57941548f25273bca00ec25)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:09.977682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:10.078230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 854/1718] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:10.078230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ec883-636f-4618-87db-64916c30ee79 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:10.078230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ec883-636f-4618-87db-64916c30ee79 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:10.078230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ec883-636f-4618-87db-64916c30ee79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:10.128038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:10.240328 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e22ec883-636f-4618-87db-64916c30ee79 None None] MFA Rules not processed for user `be379e1a6add42b69e9fdd676efca280`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:10.241467 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a81a64cb57941548f25273bca00ec25', 'request_id': 'req-b16be209-88ba-4643-af51-94dd1590df1f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:10.248534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e22ec883-636f-4618-87db-64916c30ee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:10.253712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a81a64cb57941548f25273bca00ec25', 'request_id': 'req-b16be209-88ba-4643-af51-94dd1590df1f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:10.301192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a81a64cb57941548f25273bca00ec25', 'request_id': 'req-b16be209-88ba-4643-af51-94dd1590df1f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:10.441010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 855/1719] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 24 07:52:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:10.441876 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ec883-636f-4618-87db-64916c30ee79 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:10.456408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] The token cache is being invalidate because project 5a81a64cb57941548f25273bca00ec25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:10.456504 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:10.460115 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:10.517733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 865/1720] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:09 2022] DELETE /identity/v3/projects/5a81a64cb57941548f25273bca00ec25 => generated 0 bytes in 858 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:10.521814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:10.524560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:10.737868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16be209-88ba-4643-af51-94dd1590df1f admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:10.745812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e9e46-72e9-491e-ac87-9ab314fe939b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:10.746906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e9e46-72e9-491e-ac87-9ab314fe939b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:10.747852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e9e46-72e9-491e-ac87-9ab314fe939b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:10.748358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e9e46-72e9-491e-ac87-9ab314fe939b admin admin] PATH_INFO: `/v3/users/8a028ff249444ab285c694723465ac72` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:10.748496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ec883-636f-4618-87db-64916c30ee79 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:10.759843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e9e46-72e9-491e-ac87-9ab314fe939b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a028ff249444ab285c694723465ac72)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:10.772140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e9e46-72e9-491e-ac87-9ab314fe939b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:10.778908 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d5ce7-4e33-4940-a2f7-6274c511c832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:10.779740 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d5ce7-4e33-4940-a2f7-6274c511c832 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:10.780127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d5ce7-4e33-4940-a2f7-6274c511c832 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:10.780505 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d5ce7-4e33-4940-a2f7-6274c511c832 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:10.781300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d5ce7-4e33-4940-a2f7-6274c511c832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:10.783394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d5ce7-4e33-4940-a2f7-6274c511c832 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:10.974873 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 856/1721] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:10 2022] POST /identity/v3/projects => generated 367 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:10.980262 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d5ce7-4e33-4940-a2f7-6274c511c832 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:10.982194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9a5d5ce7-4e33-4940-a2f7-6274c511c832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:11.080958 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b63e9e46-72e9-491e-ac87-9ab314fe939b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a028ff249444ab285c694723465ac72'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.093786 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b63e9e46-72e9-491e-ac87-9ab314fe939b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a028ff249444ab285c694723465ac72'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.126540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:11.171329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b63e9e46-72e9-491e-ac87-9ab314fe939b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a028ff249444ab285c694723465ac72'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.172000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b63e9e46-72e9-491e-ac87-9ab314fe939b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a028ff249444ab285c694723465ac72'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.177462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d5ce7-4e33-4940-a2f7-6274c511c832 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:11.191378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c848a-fa39-49b3-93b0-b5be47defaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:11.192039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c848a-fa39-49b3-93b0-b5be47defaa1 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:11.192426 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c848a-fa39-49b3-93b0-b5be47defaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:11.192524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b63e9e46-72e9-491e-ac87-9ab314fe939b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a028ff249444ab285c694723465ac72'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:11.192609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c848a-fa39-49b3-93b0-b5be47defaa1 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:11.194845 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c848a-fa39-49b3-93b0-b5be47defaa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:11.200929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c848a-fa39-49b3-93b0-b5be47defaa1 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:11.227081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 866/1722] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:10 2022] DELETE /identity/v3/users/8a028ff249444ab285c694723465ac72 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:11.228298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e9e46-72e9-491e-ac87-9ab314fe939b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:11.236607 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b63e9e46-72e9-491e-ac87-9ab314fe939b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:11.304902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 857/1723] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:10 2022] POST /identity/v3/users => generated 290 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:52:11.311863 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c848a-fa39-49b3-93b0-b5be47defaa1 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:11.311863 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fe5c848a-fa39-49b3-93b0-b5be47defaa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:11.337428 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c848a-fa39-49b3-93b0-b5be47defaa1 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:11.400021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e9e46-72e9-491e-ac87-9ab314fe939b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:11.406552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f236372-713b-47ab-99f1-9201e87c98f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b629ad3e6b9f4723b103f9845df2aaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:11.407576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f236372-713b-47ab-99f1-9201e87c98f7 service swift] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:11.408067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f236372-713b-47ab-99f1-9201e87c98f7 service swift] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:11.408545 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f236372-713b-47ab-99f1-9201e87c98f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:11.436826 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3f236372-713b-47ab-99f1-9201e87c98f7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:11.547881 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:11.599094 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf886a-a820-4d46-80c7-363d12e894c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:11.600660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf886a-a820-4d46-80c7-363d12e894c7 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:11.600660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf886a-a820-4d46-80c7-363d12e894c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:11.600660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf886a-a820-4d46-80c7-363d12e894c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:11.601487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf886a-a820-4d46-80c7-363d12e894c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:11.603122 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf886a-a820-4d46-80c7-363d12e894c7 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:11.663096 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 858/1724] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:11.664062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf886a-a820-4d46-80c7-363d12e894c7 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:11.697079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf886a-a820-4d46-80c7-363d12e894c7 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:11.700382 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f236372-713b-47ab-99f1-9201e87c98f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:11.709063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f236372-713b-47ab-99f1-9201e87c98f7 service swift] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:11.725602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcc1b4-3aab-47b7-ba36-542817bcc653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:11.726249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc1b4-3aab-47b7-ba36-542817bcc653 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:11.726842 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc1b4-3aab-47b7-ba36-542817bcc653 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:11.727644 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc1b4-3aab-47b7-ba36-542817bcc653 admin admin] PATH_INFO: `/v3/projects/7486e9970a164461abc6e261836d38e4/users/2de17489d28843568560d03105b65637/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:11.741013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 867/1725] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 24 07:52:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:52:11.781836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bcc1b4-3aab-47b7-ba36-542817bcc653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7486e9970a164461abc6e261836d38e4, user_id=2de17489d28843568560d03105b65637, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:11.797519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bcc1b4-3aab-47b7-ba36-542817bcc653 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:11.961932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 859/1726] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:11 2022] PUT /identity/v3/projects/7486e9970a164461abc6e261836d38e4/users/2de17489d28843568560d03105b65637/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:11.967182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f236372-713b-47ab-99f1-9201e87c98f7 service swift] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:11.995604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f236372-713b-47ab-99f1-9201e87c98f7 service swift] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:12.236783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5af6c-659f-4503-b187-b5adca642c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:12.238226 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5af6c-659f-4503-b187-b5adca642c84 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:12.238682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5af6c-659f-4503-b187-b5adca642c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:12.239026 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5af6c-659f-4503-b187-b5adca642c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:12.239902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5af6c-659f-4503-b187-b5adca642c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:12.242707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5af6c-659f-4503-b187-b5adca642c84 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:12.256678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 868/1727] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:52:12.261135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcc1b4-3aab-47b7-ba36-542817bcc653 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:12.287280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcc1b4-3aab-47b7-ba36-542817bcc653 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:12.315924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75323ef3-47a1-4b5a-817e-5114a81f99cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:12.316613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75323ef3-47a1-4b5a-817e-5114a81f99cf admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:12.316962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75323ef3-47a1-4b5a-817e-5114a81f99cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:12.317242 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75323ef3-47a1-4b5a-817e-5114a81f99cf admin admin] PATH_INFO: `/v3/projects/7486e9970a164461abc6e261836d38e4/users/2de17489d28843568560d03105b65637/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:12.323444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5af6c-659f-4503-b187-b5adca642c84 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:12.328299 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-23e5af6c-659f-4503-b187-b5adca642c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:12.330919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75323ef3-47a1-4b5a-817e-5114a81f99cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7486e9970a164461abc6e261836d38e4, user_id=2de17489d28843568560d03105b65637, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:12.336744 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75323ef3-47a1-4b5a-817e-5114a81f99cf admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:12.386129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 860/1728] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:12 2022] PUT /identity/v3/projects/7486e9970a164461abc6e261836d38e4/users/2de17489d28843568560d03105b65637/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:12.387888 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293c4b6-b657-482c-9467-da1059b320ea None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:12.388146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293c4b6-b657-482c-9467-da1059b320ea None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:12.388371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293c4b6-b657-482c-9467-da1059b320ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:12.403511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5af6c-659f-4503-b187-b5adca642c84 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:12.461755 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-9293c4b6-b657-482c-9467-da1059b320ea None None] MFA Rules not processed for user `2de17489d28843568560d03105b65637`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:12.469483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9293c4b6-b657-482c-9467-da1059b320ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:12.521487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:12.522488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:12.523139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:12.523793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc admin admin] PATH_INFO: `/v3/projects/ec3c6f87e57a4523a3c45121c5df55e8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:12.525344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec3c6f87e57a4523a3c45121c5df55e8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:12.528346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:12.637499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 861/1729] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:12.650381 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293c4b6-b657-482c-9467-da1059b320ea None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:12.654765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9293c4b6-b657-482c-9467-da1059b320ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:12.706100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec3c6f87e57a4523a3c45121c5df55e8', 'request_id': 'req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:12.712330 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec3c6f87e57a4523a3c45121c5df55e8', 'request_id': 'req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:12.718818 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec3c6f87e57a4523a3c45121c5df55e8', 'request_id': 'req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:12.721112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:12.726465 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293c4b6-b657-482c-9467-da1059b320ea None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:12.828601 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc admin admin] The token cache is being invalidate because project ec3c6f87e57a4523a3c45121c5df55e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:12.828925 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:12.833744 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b50c5cd7-a964-4795-b8b2-1b2d1db2cedc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:12.837041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 869/1730] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:12 2022] DELETE /identity/v3/projects/ec3c6f87e57a4523a3c45121c5df55e8 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:12.840095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157325d3-8937-47f1-8bdb-ca7b2dd294ab None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:12.840431 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157325d3-8937-47f1-8bdb-ca7b2dd294ab None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:12.840841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157325d3-8937-47f1-8bdb-ca7b2dd294ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:12.841314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d5ce2-7ea4-4397-a8d8-fa3a3ae57dd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:12.844414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d5ce2-7ea4-4397-a8d8-fa3a3ae57dd1 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:12.844936 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d5ce2-7ea4-4397-a8d8-fa3a3ae57dd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:12.845414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d5ce2-7ea4-4397-a8d8-fa3a3ae57dd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:12.853318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-463d5ce2-7ea4-4397-a8d8-fa3a3ae57dd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:12.897888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-157325d3-8937-47f1-8bdb-ca7b2dd294ab None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:12.899122 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-157325d3-8937-47f1-8bdb-ca7b2dd294ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:12.964079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d5ce2-7ea4-4397-a8d8-fa3a3ae57dd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:12.966848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d5ce2-7ea4-4397-a8d8-fa3a3ae57dd1 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:12.981737 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 862/1731] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:13.004701 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:13.021217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 870/1732] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:13.032955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c467d-d407-48c2-84e6-52648e48916e None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:13.033254 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c467d-d407-48c2-84e6-52648e48916e None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:13.033597 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c467d-d407-48c2-84e6-52648e48916e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:13.076823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e38c467d-d407-48c2-84e6-52648e48916e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:13.082560 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e38c467d-d407-48c2-84e6-52648e48916e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:13.094365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 863/1733] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:13.097925 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157325d3-8937-47f1-8bdb-ca7b2dd294ab None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:13.109078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157325d3-8937-47f1-8bdb-ca7b2dd294ab None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:13.114076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f746d09-fa17-4458-8d0d-087acdb3f523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:13.114718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f746d09-fa17-4458-8d0d-087acdb3f523 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:13.115003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f746d09-fa17-4458-8d0d-087acdb3f523 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:13.115274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f746d09-fa17-4458-8d0d-087acdb3f523 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:13.116082 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f746d09-fa17-4458-8d0d-087acdb3f523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:13.116557 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f746d09-fa17-4458-8d0d-087acdb3f523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:13.118198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f746d09-fa17-4458-8d0d-087acdb3f523 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:13.131759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 871/1734] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:52:13.136267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c467d-d407-48c2-84e6-52648e48916e None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:13.150514 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c467d-d407-48c2-84e6-52648e48916e None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:13.162227 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1be37-b426-438a-8529-fe3d7fb41a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:13.162850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1be37-b426-438a-8529-fe3d7fb41a2d admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:13.163135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1be37-b426-438a-8529-fe3d7fb41a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:13.163439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1be37-b426-438a-8529-fe3d7fb41a2d admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:13.164086 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1be37-b426-438a-8529-fe3d7fb41a2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:13.165773 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f746d09-fa17-4458-8d0d-087acdb3f523 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:13.166857 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1be37-b426-438a-8529-fe3d7fb41a2d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:13.167939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5f746d09-fa17-4458-8d0d-087acdb3f523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:13.183667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f746d09-fa17-4458-8d0d-087acdb3f523 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:13.187982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f529034-12c0-45aa-9a13-a4b8d6c4cac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:13.188679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f529034-12c0-45aa-9a13-a4b8d6c4cac3 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:13.189098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f529034-12c0-45aa-9a13-a4b8d6c4cac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:13.189385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f529034-12c0-45aa-9a13-a4b8d6c4cac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:13.191562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7f529034-12c0-45aa-9a13-a4b8d6c4cac3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:13.317030 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f529034-12c0-45aa-9a13-a4b8d6c4cac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:13.318764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f529034-12c0-45aa-9a13-a4b8d6c4cac3 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:13.329165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 872/1735] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:13.358769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 864/1736] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:13 2022] POST /identity/v3/projects => generated 379 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:13.358937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f529034-12c0-45aa-9a13-a4b8d6c4cac3 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:13.379720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f529034-12c0-45aa-9a13-a4b8d6c4cac3 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:13.446864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb3f8e-fcb0-438e-9bb8-3cab1ca08a7b None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:13.447560 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb3f8e-fcb0-438e-9bb8-3cab1ca08a7b None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:13.449016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb3f8e-fcb0-438e-9bb8-3cab1ca08a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:13.486160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-76bb3f8e-fcb0-438e-9bb8-3cab1ca08a7b None None] MFA Rules not processed for user `2ec3daa674814619976aead75459e98c`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:13.487959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-76bb3f8e-fcb0-438e-9bb8-3cab1ca08a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:13.609237 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e3ef7-3a3f-4ec7-865e-184b24ccb09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:13.609237 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e3ef7-3a3f-4ec7-865e-184b24ccb09f admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:13.609237 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e3ef7-3a3f-4ec7-865e-184b24ccb09f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:13.610083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e3ef7-3a3f-4ec7-865e-184b24ccb09f admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:13.614376 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e3ef7-3a3f-4ec7-865e-184b24ccb09f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:13.630476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e3ef7-3a3f-4ec7-865e-184b24ccb09f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:13.634100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 865/1737] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:13.802644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 873/1738] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:13 2022] POST /identity/v3/users => generated 290 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:13.807177 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb3f8e-fcb0-438e-9bb8-3cab1ca08a7b None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:13.838605 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb3f8e-fcb0-438e-9bb8-3cab1ca08a7b None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:13.868547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403776de-5592-4b13-9654-f4a858c5f1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:13.902634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403776de-5592-4b13-9654-f4a858c5f1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:13.902634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403776de-5592-4b13-9654-f4a858c5f1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:13.902634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403776de-5592-4b13-9654-f4a858c5f1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:13.902634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403776de-5592-4b13-9654-f4a858c5f1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:13.902634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403776de-5592-4b13-9654-f4a858c5f1d1 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:13.909577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 866/1739] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:13.910927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e3ef7-3a3f-4ec7-865e-184b24ccb09f admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:13.931101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e3ef7-3a3f-4ec7-865e-184b24ccb09f admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:13.936399 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac683db-516e-4d72-88ea-af2cf6990f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:13.937815 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac683db-516e-4d72-88ea-af2cf6990f05 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:13.938149 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac683db-516e-4d72-88ea-af2cf6990f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:13.938427 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac683db-516e-4d72-88ea-af2cf6990f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:13.939604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac683db-516e-4d72-88ea-af2cf6990f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:13.941282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac683db-516e-4d72-88ea-af2cf6990f05 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:13.972903 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 874/1740] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:13.974229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403776de-5592-4b13-9654-f4a858c5f1d1 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:13.999511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403776de-5592-4b13-9654-f4a858c5f1d1 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:14.005867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af4d92-48e4-4004-bbc8-38898c7ab098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:14.011924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af4d92-48e4-4004-bbc8-38898c7ab098 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:14.011924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af4d92-48e4-4004-bbc8-38898c7ab098 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:14.012321 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af4d92-48e4-4004-bbc8-38898c7ab098 admin admin] PATH_INFO: `/v3/projects/4d260c3578204fc9b7f144ff54eeb6b4/users/c6c421d872064c10983dda1697c90591/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:14.064543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af4d92-48e4-4004-bbc8-38898c7ab098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d260c3578204fc9b7f144ff54eeb6b4, user_id=c6c421d872064c10983dda1697c90591, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:14.067567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af4d92-48e4-4004-bbc8-38898c7ab098 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:14.081333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd001be-4b70-47c1-a43e-d62ffdbb13ce None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:14.081687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd001be-4b70-47c1-a43e-d62ffdbb13ce None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:14.082025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd001be-4b70-47c1-a43e-d62ffdbb13ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:14.120138 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9dd001be-4b70-47c1-a43e-d62ffdbb13ce None None] MFA Rules not processed for user `46e61402c01c4f9ba5f18e94960bc313`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:14.126424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9dd001be-4b70-47c1-a43e-d62ffdbb13ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:14.212164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:14.221301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 867/1741] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:13 2022] PUT /identity/v3/projects/4d260c3578204fc9b7f144ff54eeb6b4/users/c6c421d872064c10983dda1697c90591/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:14.238869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbd782-ebf3-4c27-881a-6a6de540866b None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:14.239704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbd782-ebf3-4c27-881a-6a6de540866b None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:14.240534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbd782-ebf3-4c27-881a-6a6de540866b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:14.289010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 875/1742] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Aug 24 07:52:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:14.323087 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-28bbd782-ebf3-4c27-881a-6a6de540866b None None] MFA Rules not processed for user `c6c421d872064c10983dda1697c90591`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:14.327243 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-28bbd782-ebf3-4c27-881a-6a6de540866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:14.407725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd001be-4b70-47c1-a43e-d62ffdbb13ce None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:14.409364 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9dd001be-4b70-47c1-a43e-d62ffdbb13ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:14.444974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd001be-4b70-47c1-a43e-d62ffdbb13ce None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:14.610696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 868/1743] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:14.610696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbd782-ebf3-4c27-881a-6a6de540866b None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:14.632312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf5814-e09b-498a-9537-c94b12f7a114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:14.633101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf5814-e09b-498a-9537-c94b12f7a114 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:14.633673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf5814-e09b-498a-9537-c94b12f7a114 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:14.634019 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf5814-e09b-498a-9537-c94b12f7a114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:14.680792 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf5814-e09b-498a-9537-c94b12f7a114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:14.685610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf5814-e09b-498a-9537-c94b12f7a114 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:14.735284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 876/1744] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:14.879231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbd782-ebf3-4c27-881a-6a6de540866b None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:14.931041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203d91e-db84-4d87-a922-fb465c0045ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:14.937561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203d91e-db84-4d87-a922-fb465c0045ff admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:14.937561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203d91e-db84-4d87-a922-fb465c0045ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:14.937561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203d91e-db84-4d87-a922-fb465c0045ff admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:14.950779 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203d91e-db84-4d87-a922-fb465c0045ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:14.957497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203d91e-db84-4d87-a922-fb465c0045ff admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:15.105304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 869/1745] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:14 2022] POST /identity/v3/projects => generated 381 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:15.110066 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf5814-e09b-498a-9537-c94b12f7a114 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:15.149467 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf5814-e09b-498a-9537-c94b12f7a114 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:15.353053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e56891-ea80-4a80-a40f-bb0978823055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:15.353901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e56891-ea80-4a80-a40f-bb0978823055 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:15.358570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e56891-ea80-4a80-a40f-bb0978823055 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:15.359178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e56891-ea80-4a80-a40f-bb0978823055 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:15.359990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e56891-ea80-4a80-a40f-bb0978823055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:15.368052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e56891-ea80-4a80-a40f-bb0978823055 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:15.667423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 877/1746] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:15 2022] POST /identity/v3/users => generated 297 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:15.668762 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203d91e-db84-4d87-a922-fb465c0045ff admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:15.668762 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203d91e-db84-4d87-a922-fb465c0045ff admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:15.668762 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b583d45-a2a9-4795-862b-265dc67d1059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:15.668762 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b583d45-a2a9-4795-862b-265dc67d1059 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:15.669738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b583d45-a2a9-4795-862b-265dc67d1059 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:15.669738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b583d45-a2a9-4795-862b-265dc67d1059 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:15.669738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b583d45-a2a9-4795-862b-265dc67d1059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:15.669738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b583d45-a2a9-4795-862b-265dc67d1059 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:15.669738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 870/1747] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:15.670166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e56891-ea80-4a80-a40f-bb0978823055 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:15.688548 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e56891-ea80-4a80-a40f-bb0978823055 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:15.719286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470d3c6-8a50-42f8-9221-504f1ab7a118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:15.720019 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470d3c6-8a50-42f8-9221-504f1ab7a118 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:15.720294 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470d3c6-8a50-42f8-9221-504f1ab7a118 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:15.720576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470d3c6-8a50-42f8-9221-504f1ab7a118 admin admin] PATH_INFO: `/v3/projects/0fdf4ee0a7b34a06ba175eb9885ceabb/users/0e0a9ea2962e4470ab3225d75980fc9b/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:15.813017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470d3c6-8a50-42f8-9221-504f1ab7a118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fdf4ee0a7b34a06ba175eb9885ceabb, user_id=0e0a9ea2962e4470ab3225d75980fc9b, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:15.828085 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470d3c6-8a50-42f8-9221-504f1ab7a118 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:15.869194 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:15.988446 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 878/1748] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:15 2022] PUT /identity/v3/projects/0fdf4ee0a7b34a06ba175eb9885ceabb/users/0e0a9ea2962e4470ab3225d75980fc9b/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:15.994873 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b583d45-a2a9-4795-862b-265dc67d1059 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:16.013368 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b583d45-a2a9-4795-862b-265dc67d1059 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:16.193695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1591ea1-e160-4b65-93bc-ca18daaaa5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:16.197160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1591ea1-e160-4b65-93bc-ca18daaaa5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:16.197160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1591ea1-e160-4b65-93bc-ca18daaaa5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:16.197160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1591ea1-e160-4b65-93bc-ca18daaaa5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:16.197160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1591ea1-e160-4b65-93bc-ca18daaaa5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:16.198851 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1591ea1-e160-4b65-93bc-ca18daaaa5f3 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:16.233236 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 871/1749] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:52:16.235367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470d3c6-8a50-42f8-9221-504f1ab7a118 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:16.246339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:16.253314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470d3c6-8a50-42f8-9221-504f1ab7a118 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:16.263049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919b012-09a3-41c7-a8cb-ecac043cec6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:16.263049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919b012-09a3-41c7-a8cb-ecac043cec6c admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:16.263049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919b012-09a3-41c7-a8cb-ecac043cec6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:16.263049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919b012-09a3-41c7-a8cb-ecac043cec6c admin admin] PATH_INFO: `/v3/projects/0fdf4ee0a7b34a06ba175eb9885ceabb/users/0e0a9ea2962e4470ab3225d75980fc9b/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:16.274837 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919b012-09a3-41c7-a8cb-ecac043cec6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fdf4ee0a7b34a06ba175eb9885ceabb, user_id=0e0a9ea2962e4470ab3225d75980fc9b, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:16.279929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919b012-09a3-41c7-a8cb-ecac043cec6c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:16.302084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 879/1750] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:16 2022] PUT /identity/v3/projects/0fdf4ee0a7b34a06ba175eb9885ceabb/users/0e0a9ea2962e4470ab3225d75980fc9b/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:16.316279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fa8ad-a7aa-4c57-886a-63b9e4a98f1f None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:16.316613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fa8ad-a7aa-4c57-886a-63b9e4a98f1f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:16.326297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fa8ad-a7aa-4c57-886a-63b9e4a98f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:16.436760 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d99fa8ad-a7aa-4c57-886a-63b9e4a98f1f None None] MFA Rules not processed for user `0e0a9ea2962e4470ab3225d75980fc9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:16.438238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d99fa8ad-a7aa-4c57-886a-63b9e4a98f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:16.746217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 872/1751] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:16 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:16.764371 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919b012-09a3-41c7-a8cb-ecac043cec6c admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:16.769625 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6919b012-09a3-41c7-a8cb-ecac043cec6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:16.942034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919b012-09a3-41c7-a8cb-ecac043cec6c admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:16.953132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80630c31-0e60-4582-ba87-a519bfdc527c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:16.958481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80630c31-0e60-4582-ba87-a519bfdc527c service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:16.958481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80630c31-0e60-4582-ba87-a519bfdc527c service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:16.958481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80630c31-0e60-4582-ba87-a519bfdc527c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:16.985456 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80630c31-0e60-4582-ba87-a519bfdc527c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:16.989699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80630c31-0e60-4582-ba87-a519bfdc527c service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:17.031360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:17.042345 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 880/1752] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:16 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:18.206083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:19.160448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fa8ad-a7aa-4c57-886a-63b9e4a98f1f None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:19.178358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fa8ad-a7aa-4c57-886a-63b9e4a98f1f None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:19.203612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d15e59-7dde-40a9-8a26-6cc5d838325a tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0a9ea2962e4470ab3225d75980fc9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0fdf4ee0a7b34a06ba175eb9885ceabb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:19.204919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d15e59-7dde-40a9-8a26-6cc5d838325a tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:19.205388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d15e59-7dde-40a9-8a26-6cc5d838325a tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:19.205858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d15e59-7dde-40a9-8a26-6cc5d838325a tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:19.206744 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d15e59-7dde-40a9-8a26-6cc5d838325a tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:19.207217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d15e59-7dde-40a9-8a26-6cc5d838325a tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:19.212898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d15e59-7dde-40a9-8a26-6cc5d838325a tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:19.302747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 873/1753] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:52:19.306677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80630c31-0e60-4582-ba87-a519bfdc527c service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:19.330504 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80630c31-0e60-4582-ba87-a519bfdc527c service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:19.340024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e21550-8b3b-4fa5-ace5-6d873d710b18 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:19.340264 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e21550-8b3b-4fa5-ace5-6d873d710b18 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:19.340547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e21550-8b3b-4fa5-ace5-6d873d710b18 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:19.342087 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 874/1754] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 24 07:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:52:19.348432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61169399-c539-4804-8d13-4e992af8624a tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0a9ea2962e4470ab3225d75980fc9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0fdf4ee0a7b34a06ba175eb9885ceabb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:19.349120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61169399-c539-4804-8d13-4e992af8624a tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:19.349411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61169399-c539-4804-8d13-4e992af8624a tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:19.349690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61169399-c539-4804-8d13-4e992af8624a tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:19.350349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61169399-c539-4804-8d13-4e992af8624a tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:19.352298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61169399-c539-4804-8d13-4e992af8624a tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:19.492575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 881/1755] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:19 2022] POST /identity/v3/projects => generated 354 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:19.497813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e21550-8b3b-4fa5-ace5-6d873d710b18 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:19.665652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97dc09-78ce-4423-b9ca-13b53925ccc9 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:19.670133 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97dc09-78ce-4423-b9ca-13b53925ccc9 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:19.671483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97dc09-78ce-4423-b9ca-13b53925ccc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:19.744825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9c97dc09-78ce-4423-b9ca-13b53925ccc9 None None] MFA Rules not processed for user `bdf81f09006a42299c0807f77f16c748`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:19.746100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9c97dc09-78ce-4423-b9ca-13b53925ccc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:19.754899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e21550-8b3b-4fa5-ace5-6d873d710b18 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:19.792447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069b443-9596-44cc-bea0-17e2742c01c6 tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0a9ea2962e4470ab3225d75980fc9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0fdf4ee0a7b34a06ba175eb9885ceabb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:19.792447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069b443-9596-44cc-bea0-17e2742c01c6 tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:19.792447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069b443-9596-44cc-bea0-17e2742c01c6 tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:19.792900 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069b443-9596-44cc-bea0-17e2742c01c6 tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:19.799123 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069b443-9596-44cc-bea0-17e2742c01c6 tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:19.809670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069b443-9596-44cc-bea0-17e2742c01c6 tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:19.809670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069b443-9596-44cc-bea0-17e2742c01c6 tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:19.858204 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 875/1756] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:52:19.968281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 882/1757] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 24 07:52:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:20.067324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069b443-9596-44cc-bea0-17e2742c01c6 tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:20.092741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069b443-9596-44cc-bea0-17e2742c01c6 tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:20.114529 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e0a9ea2962e4470ab3225d75980fc9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0fdf4ee0a7b34a06ba175eb9885ceabb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:20.124783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:20.124783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:20.124783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] PATH_INFO: `/v3/projects/4f3cb564a1fc434691ed089e6c8728e8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:20.124783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4f3cb564a1fc434691ed089e6c8728e8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:20.125559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:20.328713 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3cb564a1fc434691ed089e6c8728e8', 'request_id': 'req-619e6de8-01cd-4d5b-92e0-046123318abb'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:20.334622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3cb564a1fc434691ed089e6c8728e8', 'request_id': 'req-619e6de8-01cd-4d5b-92e0-046123318abb'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:20.337248 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97dc09-78ce-4423-b9ca-13b53925ccc9 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:20.338760 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9c97dc09-78ce-4423-b9ca-13b53925ccc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:20.355138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:20.451054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3cb564a1fc434691ed089e6c8728e8', 'request_id': 'req-619e6de8-01cd-4d5b-92e0-046123318abb'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:20.468375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] The token cache is being invalidate because project 4f3cb564a1fc434691ed089e6c8728e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:20.468739 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:20.469467 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:20.471288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 876/1758] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:20 2022] DELETE /identity/v3/projects/4f3cb564a1fc434691ed089e6c8728e8 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:20.477452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:20.479147 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:20.578862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97dc09-78ce-4423-b9ca-13b53925ccc9 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:20.608500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:20.622896 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e6de8-01cd-4d5b-92e0-046123318abb tempest-VolumeQuotaClassesTest-1142545042 tempest-VolumeQuotaClassesTest-1142545042-project-admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:20.639471 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4551453-7bad-4590-97d3-8a77faa63957 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:20.640884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4551453-7bad-4590-97d3-8a77faa63957 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:20.641371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4551453-7bad-4590-97d3-8a77faa63957 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:20.642778 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4551453-7bad-4590-97d3-8a77faa63957 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:20.648965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f4551453-7bad-4590-97d3-8a77faa63957 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:20.664012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4551453-7bad-4590-97d3-8a77faa63957 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:20.665853 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4551453-7bad-4590-97d3-8a77faa63957 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:20.690566 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 877/1759] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:20.713471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814402d5-37a6-4ace-b1f1-96578037d0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:20.714412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814402d5-37a6-4ace-b1f1-96578037d0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:20.714857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814402d5-37a6-4ace-b1f1-96578037d0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:20.715352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814402d5-37a6-4ace-b1f1-96578037d0ce admin admin] PATH_INFO: `/v3/users/27325b962f1e4009ad6f12d09514a7b2` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:20.717854 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814402d5-37a6-4ace-b1f1-96578037d0ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=27325b962f1e4009ad6f12d09514a7b2)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:20.720674 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814402d5-37a6-4ace-b1f1-96578037d0ce admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:21.034010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-814402d5-37a6-4ace-b1f1-96578037d0ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27325b962f1e4009ad6f12d09514a7b2'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:21.048669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-814402d5-37a6-4ace-b1f1-96578037d0ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27325b962f1e4009ad6f12d09514a7b2'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:21.142833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-814402d5-37a6-4ace-b1f1-96578037d0ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27325b962f1e4009ad6f12d09514a7b2'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:21.143441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-814402d5-37a6-4ace-b1f1-96578037d0ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27325b962f1e4009ad6f12d09514a7b2'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:21.175020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-814402d5-37a6-4ace-b1f1-96578037d0ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27325b962f1e4009ad6f12d09514a7b2'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:21.189402 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 883/1760] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:20 2022] DELETE /identity/v3/users/27325b962f1e4009ad6f12d09514a7b2 => generated 0 bytes in 852 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:21.195058 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7f17-56f4-4732-876d-3ea3a3f06670 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:21.195582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7f17-56f4-4732-876d-3ea3a3f06670 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:21.195969 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7f17-56f4-4732-876d-3ea3a3f06670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:21.265168 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-befc7f17-56f4-4732-876d-3ea3a3f06670 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:21.267027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-befc7f17-56f4-4732-876d-3ea3a3f06670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:21.396165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814402d5-37a6-4ace-b1f1-96578037d0ce admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:21.400522 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-814402d5-37a6-4ace-b1f1-96578037d0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:21.420182 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 878/1761] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:21.426130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814402d5-37a6-4ace-b1f1-96578037d0ce admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:21.434835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc7f17-56f4-4732-876d-3ea3a3f06670 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:21.434984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06897370-9b7f-41bb-9f3c-c776fc284f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:21.435820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06897370-9b7f-41bb-9f3c-c776fc284f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:21.437112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-befc7f17-56f4-4732-876d-3ea3a3f06670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:21.442789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06897370-9b7f-41bb-9f3c-c776fc284f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:21.443633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06897370-9b7f-41bb-9f3c-c776fc284f4f admin admin] PATH_INFO: `/v3/users/5fe090e27b2c4669b92a73dd364deb2c` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:21.445396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06897370-9b7f-41bb-9f3c-c776fc284f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fe090e27b2c4669b92a73dd364deb2c)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:21.447949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06897370-9b7f-41bb-9f3c-c776fc284f4f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:21.556661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc7f17-56f4-4732-876d-3ea3a3f06670 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:21.562091 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d59c52-44eb-4929-87a9-65a81cf5d240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:21.562721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d59c52-44eb-4929-87a9-65a81cf5d240 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:21.563201 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d59c52-44eb-4929-87a9-65a81cf5d240 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:21.563686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d59c52-44eb-4929-87a9-65a81cf5d240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:21.588051 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d59c52-44eb-4929-87a9-65a81cf5d240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:21.595713 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d59c52-44eb-4929-87a9-65a81cf5d240 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:21.616957 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 879/1762] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:21.617841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d59c52-44eb-4929-87a9-65a81cf5d240 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:21.622504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b6d59c52-44eb-4929-87a9-65a81cf5d240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:21.658780 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d59c52-44eb-4929-87a9-65a81cf5d240 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:21.674210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c7aac-f1a8-4056-91e5-42a98d2e3146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:21.674864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c7aac-f1a8-4056-91e5-42a98d2e3146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:21.675148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c7aac-f1a8-4056-91e5-42a98d2e3146 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:21.675424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c7aac-f1a8-4056-91e5-42a98d2e3146 admin admin] PATH_INFO: `/v3/users/c6c421d872064c10983dda1697c90591` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:21.676793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c7aac-f1a8-4056-91e5-42a98d2e3146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c421d872064c10983dda1697c90591)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:21.678918 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c7aac-f1a8-4056-91e5-42a98d2e3146 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:21.693934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:21.711461 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-06897370-9b7f-41bb-9f3c-c776fc284f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fe090e27b2c4669b92a73dd364deb2c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:21.722652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-06897370-9b7f-41bb-9f3c-c776fc284f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fe090e27b2c4669b92a73dd364deb2c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:21.841470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-06897370-9b7f-41bb-9f3c-c776fc284f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fe090e27b2c4669b92a73dd364deb2c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:21.844428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-06897370-9b7f-41bb-9f3c-c776fc284f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fe090e27b2c4669b92a73dd364deb2c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:21.879408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-06897370-9b7f-41bb-9f3c-c776fc284f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fe090e27b2c4669b92a73dd364deb2c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:21.893392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 884/1763] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:21 2022] DELETE /identity/v3/users/5fe090e27b2c4669b92a73dd364deb2c => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:21.905720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1291689-2e35-4cc0-ad61-cd99244ef780 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:21.905720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1291689-2e35-4cc0-ad61-cd99244ef780 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:21.905720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1291689-2e35-4cc0-ad61-cd99244ef780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:21.963461 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a1291689-2e35-4cc0-ad61-cd99244ef780 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:21.966947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-947c7aac-f1a8-4056-91e5-42a98d2e3146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c421d872064c10983dda1697c90591'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:21.969243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a1291689-2e35-4cc0-ad61-cd99244ef780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:21.988094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:22.068163 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-947c7aac-f1a8-4056-91e5-42a98d2e3146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c421d872064c10983dda1697c90591'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:22.068930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-947c7aac-f1a8-4056-91e5-42a98d2e3146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c421d872064c10983dda1697c90591'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:22.081587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-947c7aac-f1a8-4056-91e5-42a98d2e3146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c421d872064c10983dda1697c90591'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:22.114467 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 885/1764] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:22.131129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-947c7aac-f1a8-4056-91e5-42a98d2e3146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c421d872064c10983dda1697c90591'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:22.149007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 880/1765] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:21 2022] DELETE /identity/v3/users/c6c421d872064c10983dda1697c90591 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:22.149103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd848b-753b-4c5e-af2b-065988b8cb38 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:22.149103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd848b-753b-4c5e-af2b-065988b8cb38 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:22.149103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd848b-753b-4c5e-af2b-065988b8cb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:22.190623 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-2ffd848b-753b-4c5e-af2b-065988b8cb38 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:22.193246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2ffd848b-753b-4c5e-af2b-065988b8cb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:22.205536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c7aac-f1a8-4056-91e5-42a98d2e3146 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:22.207061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-947c7aac-f1a8-4056-91e5-42a98d2e3146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:22.351762 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c7aac-f1a8-4056-91e5-42a98d2e3146 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:22.357401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 886/1766] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:22.360961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1873bd1-c3ce-40e2-a86c-ce4077fe9e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:22.361829 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873bd1-c3ce-40e2-a86c-ce4077fe9e6e service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:22.362277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873bd1-c3ce-40e2-a86c-ce4077fe9e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:22.362704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873bd1-c3ce-40e2-a86c-ce4077fe9e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:22.374150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f1873bd1-c3ce-40e2-a86c-ce4077fe9e6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:22.391059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd848b-753b-4c5e-af2b-065988b8cb38 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:22.392635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2ffd848b-753b-4c5e-af2b-065988b8cb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:22.397057 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1873bd1-c3ce-40e2-a86c-ce4077fe9e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:22.398545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1873bd1-c3ce-40e2-a86c-ce4077fe9e6e service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:22.423742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 881/1767] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:22.428152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd848b-753b-4c5e-af2b-065988b8cb38 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:22.451666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ba916-e57d-47c4-b78b-e4040e47c023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:22.453694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ba916-e57d-47c4-b78b-e4040e47c023 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:22.455772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ba916-e57d-47c4-b78b-e4040e47c023 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:22.456103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ba916-e57d-47c4-b78b-e4040e47c023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:22.486041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ba916-e57d-47c4-b78b-e4040e47c023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:22.486041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ba916-e57d-47c4-b78b-e4040e47c023 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:22.560371 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 887/1768] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:23.220787 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0a808-07d7-48fc-b3fe-1352c2ab8d7d None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:23.221202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0a808-07d7-48fc-b3fe-1352c2ab8d7d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:23.221576 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0a808-07d7-48fc-b3fe-1352c2ab8d7d None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:23.223459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 882/1769] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 24 07:52:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:52:23.306503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047c61-5161-43e8-8402-38d0f1333542 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:23.307307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047c61-5161-43e8-8402-38d0f1333542 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:23.307792 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047c61-5161-43e8-8402-38d0f1333542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:23.347126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-30047c61-5161-43e8-8402-38d0f1333542 None None] MFA Rules not processed for user `bdf81f09006a42299c0807f77f16c748`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:23.348586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-30047c61-5161-43e8-8402-38d0f1333542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:23.469022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 888/1770] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 24 07:52:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:23.518782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0a808-07d7-48fc-b3fe-1352c2ab8d7d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:23.524144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-88d0a808-07d7-48fc-b3fe-1352c2ab8d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:23.572974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0a808-07d7-48fc-b3fe-1352c2ab8d7d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:23.585962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c8330-f5cf-4e3d-865c-44b547bd18b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:23.587940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c8330-f5cf-4e3d-865c-44b547bd18b1 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:23.589066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c8330-f5cf-4e3d-865c-44b547bd18b1 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:23.590245 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c8330-f5cf-4e3d-865c-44b547bd18b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:23.631003 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c8330-f5cf-4e3d-865c-44b547bd18b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:23.636573 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c8330-f5cf-4e3d-865c-44b547bd18b1 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:23.676615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 883/1771] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:23.787459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30047c61-5161-43e8-8402-38d0f1333542 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:23.790316 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-30047c61-5161-43e8-8402-38d0f1333542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:23.866112 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30047c61-5161-43e8-8402-38d0f1333542 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:23.883315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:23.886665 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:23.887235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:23.890665 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] PATH_INFO: `/v3/projects/ed0b02b7f3e14eb699f122372bf5aa25` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:23.894311 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed0b02b7f3e14eb699f122372bf5aa25)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:23.913143 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:24.000780 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:24.135556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed0b02b7f3e14eb699f122372bf5aa25', 'request_id': 'req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.142921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed0b02b7f3e14eb699f122372bf5aa25', 'request_id': 'req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.149654 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed0b02b7f3e14eb699f122372bf5aa25', 'request_id': 'req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.256193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] The token cache is being invalidate because project ed0b02b7f3e14eb699f122372bf5aa25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:24.256585 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.257562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.263973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 889/1772] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:23 2022] DELETE /identity/v3/projects/ed0b02b7f3e14eb699f122372bf5aa25 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:24.265724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c8330-f5cf-4e3d-865c-44b547bd18b1 service nova] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:24.281136 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c8330-f5cf-4e3d-865c-44b547bd18b1 service nova] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:24.301206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:24.313346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:24.398785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ce2a5-c72d-4f92-8d45-485ae7a1dc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:24.399480 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ce2a5-c72d-4f92-8d45-485ae7a1dc49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:24.399944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ce2a5-c72d-4f92-8d45-485ae7a1dc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:24.400416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ce2a5-c72d-4f92-8d45-485ae7a1dc49 admin admin] PATH_INFO: `/v3/users/98ff073a3a3c4fd6bfe18140536500b1` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:24.401873 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ce2a5-c72d-4f92-8d45-485ae7a1dc49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ff073a3a3c4fd6bfe18140536500b1)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:24.404853 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ce2a5-c72d-4f92-8d45-485ae7a1dc49 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:24.421531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:24.427371 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46576529-bd54-43c6-b9fe-8a5ec9b5d0d2 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:24.433790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:24.435370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:24.440856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:24.440856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] PATH_INFO: `/v3/projects/b7c01c0b053f4087985778f55bc0345f` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:24.440856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7c01c0b053f4087985778f55bc0345f)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:24.440856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:24.644584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7c01c0b053f4087985778f55bc0345f', 'request_id': 'req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.646472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b63ce2a5-c72d-4f92-8d45-485ae7a1dc49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ff073a3a3c4fd6bfe18140536500b1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.653881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7c01c0b053f4087985778f55bc0345f', 'request_id': 'req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.659911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7c01c0b053f4087985778f55bc0345f', 'request_id': 'req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.742389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b63ce2a5-c72d-4f92-8d45-485ae7a1dc49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ff073a3a3c4fd6bfe18140536500b1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.743313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b63ce2a5-c72d-4f92-8d45-485ae7a1dc49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ff073a3a3c4fd6bfe18140536500b1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.756011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b63ce2a5-c72d-4f92-8d45-485ae7a1dc49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ff073a3a3c4fd6bfe18140536500b1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.781034 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b63ce2a5-c72d-4f92-8d45-485ae7a1dc49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ff073a3a3c4fd6bfe18140536500b1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.789168 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 884/1773] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:24 2022] DELETE /identity/v3/users/98ff073a3a3c4fd6bfe18140536500b1 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:24.793542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ce2a5-c72d-4f92-8d45-485ae7a1dc49 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:24.795251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b63ce2a5-c72d-4f92-8d45-485ae7a1dc49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:24.830679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] The token cache is being invalidate because project b7c01c0b053f4087985778f55bc0345f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:24.831065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.832362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:24.835162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 890/1774] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:24 2022] DELETE /identity/v3/projects/b7c01c0b053f4087985778f55bc0345f => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:24.842025 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:24.842300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:24.898032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ce2a5-c72d-4f92-8d45-485ae7a1dc49 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:24.906115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:24.907435 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:24.907733 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:24.908011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] PATH_INFO: `/v3/projects/4d260c3578204fc9b7f144ff54eeb6b4` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:24.909323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d260c3578204fc9b7f144ff54eeb6b4)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:24.911764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:24.927149 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:24.952807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fb09d-3342-4bfc-9dfc-e26134fff8ea admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:24.958014 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e586f-4655-4618-b14f-b1ce5330e8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:24.958669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e586f-4655-4618-b14f-b1ce5330e8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:24.958952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e586f-4655-4618-b14f-b1ce5330e8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:24.959226 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e586f-4655-4618-b14f-b1ce5330e8e1 admin admin] PATH_INFO: `/v3/users/2de17489d28843568560d03105b65637` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:24.960390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e586f-4655-4618-b14f-b1ce5330e8e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2de17489d28843568560d03105b65637)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:24.962485 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e586f-4655-4618-b14f-b1ce5330e8e1 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:25.060121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d260c3578204fc9b7f144ff54eeb6b4', 'request_id': 'req-c0b12d88-277a-4cee-9dd0-abc3180df342'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.070496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d260c3578204fc9b7f144ff54eeb6b4', 'request_id': 'req-c0b12d88-277a-4cee-9dd0-abc3180df342'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.144325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-763e586f-4655-4618-b14f-b1ce5330e8e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2de17489d28843568560d03105b65637'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.151035 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-763e586f-4655-4618-b14f-b1ce5330e8e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2de17489d28843568560d03105b65637'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.170879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d260c3578204fc9b7f144ff54eeb6b4', 'request_id': 'req-c0b12d88-277a-4cee-9dd0-abc3180df342'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.181894 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] The token cache is being invalidate because project 4d260c3578204fc9b7f144ff54eeb6b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:25.182184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.182829 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.185062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 885/1775] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:24 2022] DELETE /identity/v3/projects/4d260c3578204fc9b7f144ff54eeb6b4 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:25.188978 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:25.190248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:25.253917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b12d88-277a-4cee-9dd0-abc3180df342 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:25.256763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-763e586f-4655-4618-b14f-b1ce5330e8e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2de17489d28843568560d03105b65637'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.257407 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-763e586f-4655-4618-b14f-b1ce5330e8e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2de17489d28843568560d03105b65637'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.266373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c5a6a-5d70-4ec6-baa8-2e553f638a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:25.266373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c5a6a-5d70-4ec6-baa8-2e553f638a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:25.266373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c5a6a-5d70-4ec6-baa8-2e553f638a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:25.266373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c5a6a-5d70-4ec6-baa8-2e553f638a87 admin admin] PATH_INFO: `/v3/users/0e0a9ea2962e4470ab3225d75980fc9b` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:25.267055 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c5a6a-5d70-4ec6-baa8-2e553f638a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e0a9ea2962e4470ab3225d75980fc9b)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:25.267055 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c5a6a-5d70-4ec6-baa8-2e553f638a87 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:25.272925 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-763e586f-4655-4618-b14f-b1ce5330e8e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2de17489d28843568560d03105b65637'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.279657 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 891/1776] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:24 2022] DELETE /identity/v3/users/2de17489d28843568560d03105b65637 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:25.283397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e586f-4655-4618-b14f-b1ce5330e8e1 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:25.284820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-763e586f-4655-4618-b14f-b1ce5330e8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:25.350719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e586f-4655-4618-b14f-b1ce5330e8e1 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:25.356778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6273a4-8c58-48d4-8d0c-c301f954de5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:25.357535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6273a4-8c58-48d4-8d0c-c301f954de5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:25.357892 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6273a4-8c58-48d4-8d0c-c301f954de5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:25.358166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6273a4-8c58-48d4-8d0c-c301f954de5b admin admin] PATH_INFO: `/v3/projects/694039dac8a044fb846b05f0d6b86bfd` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:25.359302 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6273a4-8c58-48d4-8d0c-c301f954de5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=694039dac8a044fb846b05f0d6b86bfd)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:25.361255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6273a4-8c58-48d4-8d0c-c301f954de5b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:25.533915 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b98c5a6a-5d70-4ec6-baa8-2e553f638a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e0a9ea2962e4470ab3225d75980fc9b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.543291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-aa6273a4-8c58-48d4-8d0c-c301f954de5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '694039dac8a044fb846b05f0d6b86bfd', 'request_id': 'req-aa6273a4-8c58-48d4-8d0c-c301f954de5b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.552697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-aa6273a4-8c58-48d4-8d0c-c301f954de5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '694039dac8a044fb846b05f0d6b86bfd', 'request_id': 'req-aa6273a4-8c58-48d4-8d0c-c301f954de5b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.560332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-aa6273a4-8c58-48d4-8d0c-c301f954de5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '694039dac8a044fb846b05f0d6b86bfd', 'request_id': 'req-aa6273a4-8c58-48d4-8d0c-c301f954de5b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.625388 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b98c5a6a-5d70-4ec6-baa8-2e553f638a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e0a9ea2962e4470ab3225d75980fc9b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.626211 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b98c5a6a-5d70-4ec6-baa8-2e553f638a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e0a9ea2962e4470ab3225d75980fc9b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.635679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b98c5a6a-5d70-4ec6-baa8-2e553f638a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e0a9ea2962e4470ab3225d75980fc9b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.650066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b98c5a6a-5d70-4ec6-baa8-2e553f638a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e0a9ea2962e4470ab3225d75980fc9b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.658567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 886/1777] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:25 2022] DELETE /identity/v3/users/0e0a9ea2962e4470ab3225d75980fc9b => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:25.705540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-aa6273a4-8c58-48d4-8d0c-c301f954de5b admin admin] The token cache is being invalidate because project 694039dac8a044fb846b05f0d6b86bfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:25.706471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-aa6273a4-8c58-48d4-8d0c-c301f954de5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.707418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-aa6273a4-8c58-48d4-8d0c-c301f954de5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:25.709409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 892/1778] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:25 2022] DELETE /identity/v3/projects/694039dac8a044fb846b05f0d6b86bfd => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:25.714338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7702626-1bc8-47ee-972d-59d31e8c2211 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:25.714626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7702626-1bc8-47ee-972d-59d31e8c2211 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:25.714899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7702626-1bc8-47ee-972d-59d31e8c2211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:25.723855 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:25.759698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e7702626-1bc8-47ee-972d-59d31e8c2211 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:25.760961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e7702626-1bc8-47ee-972d-59d31e8c2211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:25.832235 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 887/1779] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:25.839841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11a3e8-cbea-4c6f-b657-551623731170 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:25.841022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11a3e8-cbea-4c6f-b657-551623731170 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:25.841309 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11a3e8-cbea-4c6f-b657-551623731170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:25.860706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7702626-1bc8-47ee-972d-59d31e8c2211 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:25.862094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-5a11a3e8-cbea-4c6f-b657-551623731170 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:25.862806 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e7702626-1bc8-47ee-972d-59d31e8c2211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:25.863326 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5a11a3e8-cbea-4c6f-b657-551623731170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:25.873652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 893/1780] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:25.881545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7702626-1bc8-47ee-972d-59d31e8c2211 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:25.884563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11a3e8-cbea-4c6f-b657-551623731170 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:25.895789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332efdb-be9e-4971-a8f3-8b24133c107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:25.895789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332efdb-be9e-4971-a8f3-8b24133c107c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:25.895789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332efdb-be9e-4971-a8f3-8b24133c107c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:25.895789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332efdb-be9e-4971-a8f3-8b24133c107c admin admin] PATH_INFO: `/v3/projects/7486e9970a164461abc6e261836d38e4` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:25.896116 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11a3e8-cbea-4c6f-b657-551623731170 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:25.896201 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332efdb-be9e-4971-a8f3-8b24133c107c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7486e9970a164461abc6e261836d38e4)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:25.898993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332efdb-be9e-4971-a8f3-8b24133c107c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:25.902115 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63d97b-a0c0-49de-9354-43c667091ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:25.903420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63d97b-a0c0-49de-9354-43c667091ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:25.903720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63d97b-a0c0-49de-9354-43c667091ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:25.904187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63d97b-a0c0-49de-9354-43c667091ae2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:25.904871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63d97b-a0c0-49de-9354-43c667091ae2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:25.905181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63d97b-a0c0-49de-9354-43c667091ae2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:25.906901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63d97b-a0c0-49de-9354-43c667091ae2 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:25.919248 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 894/1781] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:52:25.923291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63d97b-a0c0-49de-9354-43c667091ae2 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:25.933239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63d97b-a0c0-49de-9354-43c667091ae2 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:25.938705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99051f9b-e708-4337-b8a8-7d299aae5eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:25.939303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99051f9b-e708-4337-b8a8-7d299aae5eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:25.939585 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99051f9b-e708-4337-b8a8-7d299aae5eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:25.939862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99051f9b-e708-4337-b8a8-7d299aae5eb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:25.940496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99051f9b-e708-4337-b8a8-7d299aae5eb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:25.942423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99051f9b-e708-4337-b8a8-7d299aae5eb3 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:25.995677 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5332efdb-be9e-4971-a8f3-8b24133c107c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7486e9970a164461abc6e261836d38e4', 'request_id': 'req-5332efdb-be9e-4971-a8f3-8b24133c107c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:26.003180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5332efdb-be9e-4971-a8f3-8b24133c107c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7486e9970a164461abc6e261836d38e4', 'request_id': 'req-5332efdb-be9e-4971-a8f3-8b24133c107c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:26.062885 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:26.087999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 895/1782] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:25 2022] POST /identity/v3/projects => generated 369 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:52:26.092283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99051f9b-e708-4337-b8a8-7d299aae5eb3 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:26.098034 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5332efdb-be9e-4971-a8f3-8b24133c107c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7486e9970a164461abc6e261836d38e4', 'request_id': 'req-5332efdb-be9e-4971-a8f3-8b24133c107c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:26.111439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5332efdb-be9e-4971-a8f3-8b24133c107c admin admin] The token cache is being invalidate because project 7486e9970a164461abc6e261836d38e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:26.111844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5332efdb-be9e-4971-a8f3-8b24133c107c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:26.112900 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5332efdb-be9e-4971-a8f3-8b24133c107c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:26.114812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 888/1783] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:25 2022] DELETE /identity/v3/projects/7486e9970a164461abc6e261836d38e4 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:26.119747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44781bd-e225-42bb-80bf-6679d5cf562c None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.123084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44781bd-e225-42bb-80bf-6679d5cf562c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.123084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44781bd-e225-42bb-80bf-6679d5cf562c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.142958 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d44781bd-e225-42bb-80bf-6679d5cf562c None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:26.144337 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d44781bd-e225-42bb-80bf-6679d5cf562c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:26.171927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99051f9b-e708-4337-b8a8-7d299aae5eb3 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:26.177397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e5dbf-5581-4026-81bc-d84a129aec28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:26.177914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e5dbf-5581-4026-81bc-d84a129aec28 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.178188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e5dbf-5581-4026-81bc-d84a129aec28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.178491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e5dbf-5581-4026-81bc-d84a129aec28 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.179097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e5dbf-5581-4026-81bc-d84a129aec28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:26.181164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e5dbf-5581-4026-81bc-d84a129aec28 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:26.212152 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 889/1784] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:26.220335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92af0b-60c8-4d8c-943f-c0d10c716543 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.220714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92af0b-60c8-4d8c-943f-c0d10c716543 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.221074 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92af0b-60c8-4d8c-943f-c0d10c716543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.253808 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-4d92af0b-60c8-4d8c-943f-c0d10c716543 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:26.255124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4d92af0b-60c8-4d8c-943f-c0d10c716543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:26.265191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 890/1785] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:26.266023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92af0b-60c8-4d8c-943f-c0d10c716543 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:26.267571 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4d92af0b-60c8-4d8c-943f-c0d10c716543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:26.282245 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92af0b-60c8-4d8c-943f-c0d10c716543 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:26.287121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 896/1786] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:26 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:52:26.287431 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:26.288300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e5dbf-5581-4026-81bc-d84a129aec28 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:26.288508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.288604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.289309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] PATH_INFO: `/v3/projects/0fdf4ee0a7b34a06ba175eb9885ceabb` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.290597 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fdf4ee0a7b34a06ba175eb9885ceabb)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:26.292877 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:26.301843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e5dbf-5581-4026-81bc-d84a129aec28 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:26.307342 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de35aa-0b7f-4be2-9dc4-815fd709ab94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:26.307889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de35aa-0b7f-4be2-9dc4-815fd709ab94 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.308170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de35aa-0b7f-4be2-9dc4-815fd709ab94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.308443 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de35aa-0b7f-4be2-9dc4-815fd709ab94 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.309365 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de35aa-0b7f-4be2-9dc4-815fd709ab94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:26.309846 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de35aa-0b7f-4be2-9dc4-815fd709ab94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:26.311462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de35aa-0b7f-4be2-9dc4-815fd709ab94 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:26.327724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 897/1787] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:52:26.328779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de35aa-0b7f-4be2-9dc4-815fd709ab94 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:26.330731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-63de35aa-0b7f-4be2-9dc4-815fd709ab94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:26.345053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de35aa-0b7f-4be2-9dc4-815fd709ab94 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:26.350027 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6057524-1693-4c6f-ab5b-581328f22bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:26.350673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6057524-1693-4c6f-ab5b-581328f22bef admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.350959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6057524-1693-4c6f-ab5b-581328f22bef admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.351230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6057524-1693-4c6f-ab5b-581328f22bef admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.352032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6057524-1693-4c6f-ab5b-581328f22bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:26.358165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6057524-1693-4c6f-ab5b-581328f22bef admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:26.369034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 898/1788] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:26.369656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6057524-1693-4c6f-ab5b-581328f22bef admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:26.379709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6057524-1693-4c6f-ab5b-581328f22bef admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:26.384405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461961f-8262-47d2-b51f-da21c788dc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:26.385039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461961f-8262-47d2-b51f-da21c788dc9f admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.385322 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461961f-8262-47d2-b51f-da21c788dc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.385596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461961f-8262-47d2-b51f-da21c788dc9f admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.393344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461961f-8262-47d2-b51f-da21c788dc9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:26.393344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461961f-8262-47d2-b51f-da21c788dc9f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:26.428507 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fdf4ee0a7b34a06ba175eb9885ceabb', 'request_id': 'req-2a4d29c6-1858-471f-b10e-f39defaa88e6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:26.435157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fdf4ee0a7b34a06ba175eb9885ceabb', 'request_id': 'req-2a4d29c6-1858-471f-b10e-f39defaa88e6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:26.499785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 899/1789] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:26 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:26.502951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461961f-8262-47d2-b51f-da21c788dc9f admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:26.519615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fdf4ee0a7b34a06ba175eb9885ceabb', 'request_id': 'req-2a4d29c6-1858-471f-b10e-f39defaa88e6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:26.522350 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:26.533286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] The token cache is being invalidate because project 0fdf4ee0a7b34a06ba175eb9885ceabb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:26.533699 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:26.534512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:26.539250 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 891/1790] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:26 2022] DELETE /identity/v3/projects/0fdf4ee0a7b34a06ba175eb9885ceabb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:26.539250 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:26.539250 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:26.567482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461961f-8262-47d2-b51f-da21c788dc9f admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:26.572246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785270e7-c811-4539-9488-b50accee2202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:26.572879 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785270e7-c811-4539-9488-b50accee2202 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.573165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785270e7-c811-4539-9488-b50accee2202 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.573440 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785270e7-c811-4539-9488-b50accee2202 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.574078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785270e7-c811-4539-9488-b50accee2202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:26.576428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785270e7-c811-4539-9488-b50accee2202 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:26.587329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 900/1791] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:26.593764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb4ae6-c933-4e4c-a907-5865f57bf586 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.593764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb4ae6-c933-4e4c-a907-5865f57bf586 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.593764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb4ae6-c933-4e4c-a907-5865f57bf586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.604974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d29c6-1858-471f-b10e-f39defaa88e6 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:26.611043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-7feb4ae6-c933-4e4c-a907-5865f57bf586 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:26.611189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4967e-6205-4eca-84fd-f9e31787f06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:26.611721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4967e-6205-4eca-84fd-f9e31787f06f admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.612006 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4967e-6205-4eca-84fd-f9e31787f06f admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.612287 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4967e-6205-4eca-84fd-f9e31787f06f admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.612733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7feb4ae6-c933-4e4c-a907-5865f57bf586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:26.612956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4967e-6205-4eca-84fd-f9e31787f06f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:26.614891 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4967e-6205-4eca-84fd-f9e31787f06f admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:26.623047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 901/1792] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:26.624028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb4ae6-c933-4e4c-a907-5865f57bf586 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:26.625307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7feb4ae6-c933-4e4c-a907-5865f57bf586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:26.638488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb4ae6-c933-4e4c-a907-5865f57bf586 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:26.643940 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76386b-4ffa-4d3c-98c5-53cb1eaa8079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:26.644627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76386b-4ffa-4d3c-98c5-53cb1eaa8079 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.644950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76386b-4ffa-4d3c-98c5-53cb1eaa8079 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.645221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76386b-4ffa-4d3c-98c5-53cb1eaa8079 admin admin] PATH_INFO: `/v3/projects/03351f5c52c24ad395c80329e5d640c0/users/ed4c355e9a0147dcaf2eb359deb4d24b/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.663129 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76386b-4ffa-4d3c-98c5-53cb1eaa8079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03351f5c52c24ad395c80329e5d640c0, user_id=ed4c355e9a0147dcaf2eb359deb4d24b, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:26.666313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76386b-4ffa-4d3c-98c5-53cb1eaa8079 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:26.700754 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 892/1793] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:26 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:26.702804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8218bb-ba38-4a7a-a7f8-e78827b2b56f None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.702804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8218bb-ba38-4a7a-a7f8-e78827b2b56f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.702804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8218bb-ba38-4a7a-a7f8-e78827b2b56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.726738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-ad8218bb-ba38-4a7a-a7f8-e78827b2b56f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:26.728114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ad8218bb-ba38-4a7a-a7f8-e78827b2b56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:26.750790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 893/1794] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:26.751654 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8218bb-ba38-4a7a-a7f8-e78827b2b56f None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:26.763856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8218bb-ba38-4a7a-a7f8-e78827b2b56f None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:26.765764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 902/1795] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:26 2022] PUT /identity/v3/projects/03351f5c52c24ad395c80329e5d640c0/users/ed4c355e9a0147dcaf2eb359deb4d24b/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:26.766771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76386b-4ffa-4d3c-98c5-53cb1eaa8079 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:26.821442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:INSERT=6,keystone:DELETE=1,keystone:SELECT=93 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:26.884371 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76386b-4ffa-4d3c-98c5-53cb1eaa8079 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:26.896008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24d60e-1769-4088-8eb7-1caf8b94b3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:26.900637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24d60e-1769-4088-8eb7-1caf8b94b3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.900637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24d60e-1769-4088-8eb7-1caf8b94b3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.900637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24d60e-1769-4088-8eb7-1caf8b94b3c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.900637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24d60e-1769-4088-8eb7-1caf8b94b3c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:26.900637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24d60e-1769-4088-8eb7-1caf8b94b3c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:26.900637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24d60e-1769-4088-8eb7-1caf8b94b3c6 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:26.914931 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 903/1796] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:52:26.916341 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d829a-487f-44b4-83ec-0e626c1fd972 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.916630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d829a-487f-44b4-83ec-0e626c1fd972 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.917522 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d829a-487f-44b4-83ec-0e626c1fd972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.940486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad85d7d-a28b-4a0c-b90d-d7d61542bdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:26.941161 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad85d7d-a28b-4a0c-b90d-d7d61542bdf5 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.941447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad85d7d-a28b-4a0c-b90d-d7d61542bdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.941728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad85d7d-a28b-4a0c-b90d-d7d61542bdf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.942394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad85d7d-a28b-4a0c-b90d-d7d61542bdf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:26.944042 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad85d7d-a28b-4a0c-b90d-d7d61542bdf5 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:26.966698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 894/1797] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:26.967625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad85d7d-a28b-4a0c-b90d-d7d61542bdf5 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:26.978517 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad85d7d-a28b-4a0c-b90d-d7d61542bdf5 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:26.986438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0f8d829a-487f-44b4-83ec-0e626c1fd972 None None] MFA Rules not processed for user `ed4c355e9a0147dcaf2eb359deb4d24b`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:26.988634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0f8d829a-487f-44b4-83ec-0e626c1fd972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:26.992104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5356346-1467-4afb-802c-0abf7ca02687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:26.992780 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5356346-1467-4afb-802c-0abf7ca02687 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:26.993076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5356346-1467-4afb-802c-0abf7ca02687 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:26.993347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5356346-1467-4afb-802c-0abf7ca02687 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:26.993996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5356346-1467-4afb-802c-0abf7ca02687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:26.996110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5356346-1467-4afb-802c-0abf7ca02687 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:27.088038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 895/1798] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:26 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:27.088846 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5356346-1467-4afb-802c-0abf7ca02687 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:27.095054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 904/1799] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:27.097543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d829a-487f-44b4-83ec-0e626c1fd972 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:27.127118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d829a-487f-44b4-83ec-0e626c1fd972 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:27.150138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:27.200532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5356346-1467-4afb-802c-0abf7ca02687 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:27.202070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7034af-8ce4-4066-b75f-cd0d54365e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:27.202708 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7034af-8ce4-4066-b75f-cd0d54365e18 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:27.202991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7034af-8ce4-4066-b75f-cd0d54365e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:27.203267 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7034af-8ce4-4066-b75f-cd0d54365e18 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:27.205811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7034af-8ce4-4066-b75f-cd0d54365e18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:27.206159 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd1bed-c22d-425b-af34-d359dfdd0472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:27.206853 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd1bed-c22d-425b-af34-d359dfdd0472 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:27.207133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd1bed-c22d-425b-af34-d359dfdd0472 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:27.207410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd1bed-c22d-425b-af34-d359dfdd0472 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:27.208130 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd1bed-c22d-425b-af34-d359dfdd0472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:27.208303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7034af-8ce4-4066-b75f-cd0d54365e18 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:27.209894 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd1bed-c22d-425b-af34-d359dfdd0472 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:27.224733 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 896/1800] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:27.226075 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd1bed-c22d-425b-af34-d359dfdd0472 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:27.237105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd1bed-c22d-425b-af34-d359dfdd0472 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:27.244255 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ee769-9322-49b8-8ec3-f4e37a060c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:27.244912 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ee769-9322-49b8-8ec3-f4e37a060c57 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:27.245203 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ee769-9322-49b8-8ec3-f4e37a060c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:27.245477 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ee769-9322-49b8-8ec3-f4e37a060c57 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:27.246217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ee769-9322-49b8-8ec3-f4e37a060c57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:27.248898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ee769-9322-49b8-8ec3-f4e37a060c57 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:27.332617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 905/1801] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:27 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:27.333746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7034af-8ce4-4066-b75f-cd0d54365e18 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:27.346277 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7034af-8ce4-4066-b75f-cd0d54365e18 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:27.365077 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ba9ee-0154-441d-90ff-d361713f01ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:27.365882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ba9ee-0154-441d-90ff-d361713f01ee admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:27.366326 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ba9ee-0154-441d-90ff-d361713f01ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:27.366671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ba9ee-0154-441d-90ff-d361713f01ee admin admin] PATH_INFO: `/v3/projects/3159f0caee954ba0bc7decc62df56123/users/f45dd4990fd24efe99ec730ca22fc7d6/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:27.367918 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 897/1802] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:27 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:27.369922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ee769-9322-49b8-8ec3-f4e37a060c57 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:27.392403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ba9ee-0154-441d-90ff-d361713f01ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=3159f0caee954ba0bc7decc62df56123, user_id=f45dd4990fd24efe99ec730ca22fc7d6, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:27.405687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ba9ee-0154-441d-90ff-d361713f01ee admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:27.411537 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:27.452778 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ee769-9322-49b8-8ec3-f4e37a060c57 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:27.457840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251e88b-c05b-4558-b48d-d00b1d7dd974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:27.459023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251e88b-c05b-4558-b48d-d00b1d7dd974 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:27.459521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251e88b-c05b-4558-b48d-d00b1d7dd974 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:27.460010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251e88b-c05b-4558-b48d-d00b1d7dd974 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:27.460886 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251e88b-c05b-4558-b48d-d00b1d7dd974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:27.462551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251e88b-c05b-4558-b48d-d00b1d7dd974 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:27.474578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 898/1803] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:27.475685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251e88b-c05b-4558-b48d-d00b1d7dd974 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:27.481889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 906/1804] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:27 2022] PUT /identity/v3/projects/3159f0caee954ba0bc7decc62df56123/users/f45dd4990fd24efe99ec730ca22fc7d6/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:27.487245 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251e88b-c05b-4558-b48d-d00b1d7dd974 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:27.493049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ba9ee-0154-441d-90ff-d361713f01ee admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:27.562038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730697f2-f717-47ec-ae2d-abfd71a77e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:27.562650 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730697f2-f717-47ec-ae2d-abfd71a77e1f admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:27.562929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730697f2-f717-47ec-ae2d-abfd71a77e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:27.563203 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730697f2-f717-47ec-ae2d-abfd71a77e1f admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:27.563831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730697f2-f717-47ec-ae2d-abfd71a77e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:27.575901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730697f2-f717-47ec-ae2d-abfd71a77e1f admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:27.578335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ba9ee-0154-441d-90ff-d361713f01ee admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:27.584173 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d817a9-91ea-4e07-b9df-7e78fb35a4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:27.584173 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d817a9-91ea-4e07-b9df-7e78fb35a4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:27.584584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d817a9-91ea-4e07-b9df-7e78fb35a4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:27.584584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d817a9-91ea-4e07-b9df-7e78fb35a4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:27.585184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d817a9-91ea-4e07-b9df-7e78fb35a4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:27.587472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d817a9-91ea-4e07-b9df-7e78fb35a4e1 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:27.598635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 907/1805] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:27.599971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57438a73-1737-4a25-8109-25307a84f19f None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:27.600284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57438a73-1737-4a25-8109-25307a84f19f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:27.600565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57438a73-1737-4a25-8109-25307a84f19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:27.632448 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-57438a73-1737-4a25-8109-25307a84f19f None None] MFA Rules not processed for user `f45dd4990fd24efe99ec730ca22fc7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:27.633728 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-57438a73-1737-4a25-8109-25307a84f19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:27.657698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 899/1806] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:27 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:27.659389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730697f2-f717-47ec-ae2d-abfd71a77e1f admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:27.671241 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730697f2-f717-47ec-ae2d-abfd71a77e1f admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:27.676519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa85016-d0e1-473b-becb-c7d33e100c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:27.677720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa85016-d0e1-473b-becb-c7d33e100c90 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:27.678264 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa85016-d0e1-473b-becb-c7d33e100c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:27.678804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa85016-d0e1-473b-becb-c7d33e100c90 admin admin] PATH_INFO: `/v3/projects/ed979d81272f4041af5178fbe7b78cf6/users/3de732cc89c842298888d7cc2dea6449/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:27.681293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:27.701650 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa85016-d0e1-473b-becb-c7d33e100c90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed979d81272f4041af5178fbe7b78cf6, user_id=3de732cc89c842298888d7cc2dea6449, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:27.705547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa85016-d0e1-473b-becb-c7d33e100c90 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:27.719962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 908/1807] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:27.721158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57438a73-1737-4a25-8109-25307a84f19f None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:27.733909 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57438a73-1737-4a25-8109-25307a84f19f None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:27.738674 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf9fb5-7198-4cda-8971-94b4824edfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:27.739315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf9fb5-7198-4cda-8971-94b4824edfe9 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:27.739746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf9fb5-7198-4cda-8971-94b4824edfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:27.740033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf9fb5-7198-4cda-8971-94b4824edfe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:27.740779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf9fb5-7198-4cda-8971-94b4824edfe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:27.742401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf9fb5-7198-4cda-8971-94b4824edfe9 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:27.756811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 909/1808] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:27.756811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf9fb5-7198-4cda-8971-94b4824edfe9 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:27.768476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf9fb5-7198-4cda-8971-94b4824edfe9 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:27.773322 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d2dcc-756a-4a36-9165-ef0fe941bb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:27.773946 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d2dcc-756a-4a36-9165-ef0fe941bb29 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:27.774231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d2dcc-756a-4a36-9165-ef0fe941bb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:27.774510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d2dcc-756a-4a36-9165-ef0fe941bb29 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:27.775140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d2dcc-756a-4a36-9165-ef0fe941bb29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:27.777102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d2dcc-756a-4a36-9165-ef0fe941bb29 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:27.811940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 900/1809] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:27 2022] PUT /identity/v3/projects/ed979d81272f4041af5178fbe7b78cf6/users/3de732cc89c842298888d7cc2dea6449/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:27.812959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa85016-d0e1-473b-becb-c7d33e100c90 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:27.880563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:27.891503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 910/1810] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:27 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:27.891503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626c27d-a3bc-4cbd-975f-fd6576a931ba None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:27.891503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626c27d-a3bc-4cbd-975f-fd6576a931ba None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:27.891916 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626c27d-a3bc-4cbd-975f-fd6576a931ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:27.919917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa85016-d0e1-473b-becb-c7d33e100c90 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:27.924811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-b626c27d-a3bc-4cbd-975f-fd6576a931ba None None] MFA Rules not processed for user `3de732cc89c842298888d7cc2dea6449`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:27.926405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b626c27d-a3bc-4cbd-975f-fd6576a931ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:27.941826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:28.014951 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946c106-b4af-43f1-97fd-99161a4a4540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:28.015908 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946c106-b4af-43f1-97fd-99161a4a4540 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.017883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946c106-b4af-43f1-97fd-99161a4a4540 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.018326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946c106-b4af-43f1-97fd-99161a4a4540 admin admin] PATH_INFO: `/v3/projects/4d17c11a2ae44d478cf677d73a99b2e8/users/bad476bf094443ff80c79a74242adc00/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.037453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 911/1811] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:28.038401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626c27d-a3bc-4cbd-975f-fd6576a931ba None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:28.040342 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946c106-b4af-43f1-97fd-99161a4a4540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d17c11a2ae44d478cf677d73a99b2e8, user_id=bad476bf094443ff80c79a74242adc00, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:28.043685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946c106-b4af-43f1-97fd-99161a4a4540 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:28.053914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626c27d-a3bc-4cbd-975f-fd6576a931ba None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:28.059989 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c656f2-baaa-4fe2-b835-bcdae8c0a6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:28.060719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c656f2-baaa-4fe2-b835-bcdae8c0a6bb admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.061071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c656f2-baaa-4fe2-b835-bcdae8c0a6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.061372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c656f2-baaa-4fe2-b835-bcdae8c0a6bb admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.062008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c656f2-baaa-4fe2-b835-bcdae8c0a6bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:28.063969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c656f2-baaa-4fe2-b835-bcdae8c0a6bb admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:28.100295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 901/1812] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:27 2022] PUT /identity/v3/projects/4d17c11a2ae44d478cf677d73a99b2e8/users/bad476bf094443ff80c79a74242adc00/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:28.104208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946c106-b4af-43f1-97fd-99161a4a4540 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:28.167459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 912/1813] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:28 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:28.169200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c656f2-baaa-4fe2-b835-bcdae8c0a6bb admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:28.185034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c656f2-baaa-4fe2-b835-bcdae8c0a6bb admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:28.274069 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946c106-b4af-43f1-97fd-99161a4a4540 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:28.281921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b52455-0bb5-4c31-95d9-17ced2fcd571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:28.282541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b52455-0bb5-4c31-95d9-17ced2fcd571 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.282820 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b52455-0bb5-4c31-95d9-17ced2fcd571 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.283093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b52455-0bb5-4c31-95d9-17ced2fcd571 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.283734 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b52455-0bb5-4c31-95d9-17ced2fcd571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:28.285712 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b52455-0bb5-4c31-95d9-17ced2fcd571 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:28.302410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891505be-f4ee-4b55-920e-396c868f5d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:28.303578 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891505be-f4ee-4b55-920e-396c868f5d25 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.304173 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891505be-f4ee-4b55-920e-396c868f5d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.304698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891505be-f4ee-4b55-920e-396c868f5d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.305659 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891505be-f4ee-4b55-920e-396c868f5d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:28.309563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891505be-f4ee-4b55-920e-396c868f5d25 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:28.336694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 913/1814] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:28.341345 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891505be-f4ee-4b55-920e-396c868f5d25 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:28.361814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891505be-f4ee-4b55-920e-396c868f5d25 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:28.377390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c20ae5-2d77-49b2-ba47-1a0b1b629eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:28.377952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c20ae5-2d77-49b2-ba47-1a0b1b629eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.378240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c20ae5-2d77-49b2-ba47-1a0b1b629eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.378519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c20ae5-2d77-49b2-ba47-1a0b1b629eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.379183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c20ae5-2d77-49b2-ba47-1a0b1b629eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:28.380946 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c20ae5-2d77-49b2-ba47-1a0b1b629eb2 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:28.390683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 902/1815] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:28 2022] POST /identity/v3/projects => generated 383 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:28.391900 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b52455-0bb5-4c31-95d9-17ced2fcd571 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:28.408704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 914/1816] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:28.409694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c20ae5-2d77-49b2-ba47-1a0b1b629eb2 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:28.423453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c20ae5-2d77-49b2-ba47-1a0b1b629eb2 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:28.490632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b52455-0bb5-4c31-95d9-17ced2fcd571 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:28.502777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec270e24-dbc6-4863-b131-b6c124eb4186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:28.503589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec270e24-dbc6-4863-b131-b6c124eb4186 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.503966 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec270e24-dbc6-4863-b131-b6c124eb4186 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.504365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec270e24-dbc6-4863-b131-b6c124eb4186 admin admin] PATH_INFO: `/v3/projects/4d17c11a2ae44d478cf677d73a99b2e8/users/bad476bf094443ff80c79a74242adc00/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.504534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:28.506562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec270e24-dbc6-4863-b131-b6c124eb4186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d17c11a2ae44d478cf677d73a99b2e8, user_id=bad476bf094443ff80c79a74242adc00, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:28.509492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec270e24-dbc6-4863-b131-b6c124eb4186 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:28.511305 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fa151-439a-4b8a-be23-a458f4cbfff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:28.513135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fa151-439a-4b8a-be23-a458f4cbfff6 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.513766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fa151-439a-4b8a-be23-a458f4cbfff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.514176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fa151-439a-4b8a-be23-a458f4cbfff6 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.514952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2fa151-439a-4b8a-be23-a458f4cbfff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:28.517321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2fa151-439a-4b8a-be23-a458f4cbfff6 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:28.528721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 903/1817] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:28 2022] PUT /identity/v3/projects/4d17c11a2ae44d478cf677d73a99b2e8/users/bad476bf094443ff80c79a74242adc00/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:28.533059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec270e24-dbc6-4863-b131-b6c124eb4186 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:28.565501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:28.602208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec270e24-dbc6-4863-b131-b6c124eb4186 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:28.607366 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e208212-62d5-466d-b7ab-a67ce3a3cb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:28.607987 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e208212-62d5-466d-b7ab-a67ce3a3cb7c admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.608298 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e208212-62d5-466d-b7ab-a67ce3a3cb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.608609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e208212-62d5-466d-b7ab-a67ce3a3cb7c admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.609315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e208212-62d5-466d-b7ab-a67ce3a3cb7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:28.609583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 915/1818] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:28 2022] POST /identity/v3/users => generated 298 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:28.611133 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec632604-dc3c-4a95-b193-cec8d110aa36 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.611358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec632604-dc3c-4a95-b193-cec8d110aa36 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.611630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec632604-dc3c-4a95-b193-cec8d110aa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.613872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e208212-62d5-466d-b7ab-a67ce3a3cb7c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:28.627661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 904/1819] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:28.627661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e208212-62d5-466d-b7ab-a67ce3a3cb7c admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:28.636157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e208212-62d5-466d-b7ab-a67ce3a3cb7c admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:28.641861 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af3163-ae75-4924-bb38-9bb643b49a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:28.642460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af3163-ae75-4924-bb38-9bb643b49a98 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.642749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af3163-ae75-4924-bb38-9bb643b49a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.643016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af3163-ae75-4924-bb38-9bb643b49a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.643647 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af3163-ae75-4924-bb38-9bb643b49a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:28.644700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-ec632604-dc3c-4a95-b193-cec8d110aa36 None None] MFA Rules not processed for user `bad476bf094443ff80c79a74242adc00`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:28.645279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af3163-ae75-4924-bb38-9bb643b49a98 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:28.645907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ec632604-dc3c-4a95-b193-cec8d110aa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:28.656699 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 905/1820] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:28.657479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af3163-ae75-4924-bb38-9bb643b49a98 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:28.666808 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af3163-ae75-4924-bb38-9bb643b49a98 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:28.671993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b288f-7238-40cf-94a0-6b1fd8932d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:28.672627 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b288f-7238-40cf-94a0-6b1fd8932d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.672935 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b288f-7238-40cf-94a0-6b1fd8932d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.673214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b288f-7238-40cf-94a0-6b1fd8932d90 admin admin] PATH_INFO: `/v3/projects/a385f27e57264b5089585f404c98b0f5/users/0432d7e636ee41edae2d1a1bff0bc690/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.687634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b288f-7238-40cf-94a0-6b1fd8932d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a385f27e57264b5089585f404c98b0f5, user_id=0432d7e636ee41edae2d1a1bff0bc690, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:28.690316 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b288f-7238-40cf-94a0-6b1fd8932d90 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:28.716618 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 916/1821] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:28.717868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec632604-dc3c-4a95-b193-cec8d110aa36 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:28.729123 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec632604-dc3c-4a95-b193-cec8d110aa36 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:28.734000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4e09f-bdb4-4f43-b683-a99596607a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:28.734653 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4e09f-bdb4-4f43-b683-a99596607a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.735113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4e09f-bdb4-4f43-b683-a99596607a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.735389 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4e09f-bdb4-4f43-b683-a99596607a52 admin admin] PATH_INFO: `/v3/projects/db9a8da245764743a8288c67d12a64d8/users/04bdfda2dede483498f42ee7dfe288d2/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.750202 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4e09f-bdb4-4f43-b683-a99596607a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db9a8da245764743a8288c67d12a64d8, user_id=04bdfda2dede483498f42ee7dfe288d2, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:28.752926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4e09f-bdb4-4f43-b683-a99596607a52 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:28.771077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 906/1822] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:28 2022] PUT /identity/v3/projects/a385f27e57264b5089585f404c98b0f5/users/0432d7e636ee41edae2d1a1bff0bc690/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:28.772052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b288f-7238-40cf-94a0-6b1fd8932d90 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:28.773678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-185b288f-7238-40cf-94a0-6b1fd8932d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:28.810857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 917/1823] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:28 2022] PUT /identity/v3/projects/db9a8da245764743a8288c67d12a64d8/users/04bdfda2dede483498f42ee7dfe288d2/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:28.812284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf3109-ae9d-4205-95b5-53da98c5b410 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.812575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf3109-ae9d-4205-95b5-53da98c5b410 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.812877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf3109-ae9d-4205-95b5-53da98c5b410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.834570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b288f-7238-40cf-94a0-6b1fd8932d90 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:28.842142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-4dbf3109-ae9d-4205-95b5-53da98c5b410 None None] MFA Rules not processed for user `0432d7e636ee41edae2d1a1bff0bc690`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:28.843354 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4dbf3109-ae9d-4205-95b5-53da98c5b410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:28.883197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701cb23-a32c-42b7-9944-acc7e62cc984 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:28.883724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701cb23-a32c-42b7-9944-acc7e62cc984 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.884002 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701cb23-a32c-42b7-9944-acc7e62cc984 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.884278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701cb23-a32c-42b7-9944-acc7e62cc984 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.909264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 918/1824] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:28.910180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf3109-ae9d-4205-95b5-53da98c5b410 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:28.942066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:28.951971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701cb23-a32c-42b7-9944-acc7e62cc984 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:28.953453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701cb23-a32c-42b7-9944-acc7e62cc984 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:28.953963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:28.963578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 907/1825] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:28.964366 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701cb23-a32c-42b7-9944-acc7e62cc984 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:28.975306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf3109-ae9d-4205-95b5-53da98c5b410 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:28.981163 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fed846-4e81-4a72-9a89-5a95ee52bfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:28.981963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fed846-4e81-4a72-9a89-5a95ee52bfa4 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:28.984727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fed846-4e81-4a72-9a89-5a95ee52bfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:28.984727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fed846-4e81-4a72-9a89-5a95ee52bfa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:28.984727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fed846-4e81-4a72-9a89-5a95ee52bfa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:28.985472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fed846-4e81-4a72-9a89-5a95ee52bfa4 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:28.999437 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 919/1826] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:29.000565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fed846-4e81-4a72-9a89-5a95ee52bfa4 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:29.010604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fed846-4e81-4a72-9a89-5a95ee52bfa4 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:29.016263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38be1c6-34b4-4e10-84da-ada0d3e41951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:29.016963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38be1c6-34b4-4e10-84da-ada0d3e41951 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:29.017246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38be1c6-34b4-4e10-84da-ada0d3e41951 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:29.017518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38be1c6-34b4-4e10-84da-ada0d3e41951 admin admin] PATH_INFO: `/v3/projects/db9a8da245764743a8288c67d12a64d8/users/04bdfda2dede483498f42ee7dfe288d2/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:29.019924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38be1c6-34b4-4e10-84da-ada0d3e41951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db9a8da245764743a8288c67d12a64d8, user_id=04bdfda2dede483498f42ee7dfe288d2, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:29.022690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38be1c6-34b4-4e10-84da-ada0d3e41951 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:29.032290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701cb23-a32c-42b7-9944-acc7e62cc984 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:29.034117 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 920/1827] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:28 2022] PUT /identity/v3/projects/db9a8da245764743a8288c67d12a64d8/users/04bdfda2dede483498f42ee7dfe288d2/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:29.045393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ea281-5403-4ba1-9992-c3837afa25b8 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:29.045682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ea281-5403-4ba1-9992-c3837afa25b8 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:29.045989 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ea281-5403-4ba1-9992-c3837afa25b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:29.078730 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-b49ea281-5403-4ba1-9992-c3837afa25b8 None None] MFA Rules not processed for user `04bdfda2dede483498f42ee7dfe288d2`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:29.080274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b49ea281-5403-4ba1-9992-c3837afa25b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:29.092416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a23742-278d-414b-9eb3-e808d02dcfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:29.093149 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a23742-278d-414b-9eb3-e808d02dcfe2 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:29.094128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a23742-278d-414b-9eb3-e808d02dcfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:29.094128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a23742-278d-414b-9eb3-e808d02dcfe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:29.094727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a23742-278d-414b-9eb3-e808d02dcfe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:29.097610 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a23742-278d-414b-9eb3-e808d02dcfe2 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:29.150591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 921/1828] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:29.161374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ea281-5403-4ba1-9992-c3837afa25b8 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:29.216076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ea281-5403-4ba1-9992-c3837afa25b8 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:29.219725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab2c0b-e394-4ad4-bf84-a2dcaf1b16cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:29.220336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab2c0b-e394-4ad4-bf84-a2dcaf1b16cb service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:29.220615 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab2c0b-e394-4ad4-bf84-a2dcaf1b16cb service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:29.220914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab2c0b-e394-4ad4-bf84-a2dcaf1b16cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:29.230237 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab2c0b-e394-4ad4-bf84-a2dcaf1b16cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:29.231718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab2c0b-e394-4ad4-bf84-a2dcaf1b16cb service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:29.241838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 922/1829] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:29.278256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 908/1830] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:28 2022] POST /identity/v3/projects => generated 377 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:29.282029 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab2c0b-e394-4ad4-bf84-a2dcaf1b16cb service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:29.292700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab2c0b-e394-4ad4-bf84-a2dcaf1b16cb service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:29.352244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:29.359161 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a23742-278d-414b-9eb3-e808d02dcfe2 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:29.361703 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86163473-2aea-4d82-9b6d-9527f49fe7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:29.362316 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86163473-2aea-4d82-9b6d-9527f49fe7bb admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:29.362725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86163473-2aea-4d82-9b6d-9527f49fe7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:29.363004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86163473-2aea-4d82-9b6d-9527f49fe7bb admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:29.363858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86163473-2aea-4d82-9b6d-9527f49fe7bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:29.365880 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86163473-2aea-4d82-9b6d-9527f49fe7bb admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:29.417516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a23742-278d-414b-9eb3-e808d02dcfe2 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:29.423523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba55264-d690-4630-b087-db29213c9aab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:29.424081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba55264-d690-4630-b087-db29213c9aab service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:29.424363 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba55264-d690-4630-b087-db29213c9aab service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:29.424660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba55264-d690-4630-b087-db29213c9aab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:29.466164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:29.487805 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba55264-d690-4630-b087-db29213c9aab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:29.489386 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba55264-d690-4630-b087-db29213c9aab service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:29.500239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 909/1831] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:29.557296 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 923/1832] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:29 2022] POST /identity/v3/users => generated 295 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:29.559642 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba55264-d690-4630-b087-db29213c9aab service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:29.572979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba55264-d690-4630-b087-db29213c9aab service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:29.580620 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3b5b5-822c-4498-b2ef-151354806131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:29.581479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3b5b5-822c-4498-b2ef-151354806131 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:29.581756 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3b5b5-822c-4498-b2ef-151354806131 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:29.582025 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3b5b5-822c-4498-b2ef-151354806131 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:29.583021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3b5b5-822c-4498-b2ef-151354806131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:29.584787 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3b5b5-822c-4498-b2ef-151354806131 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:29.598797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 910/1833] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:29.603115 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86163473-2aea-4d82-9b6d-9527f49fe7bb admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:29.616015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86163473-2aea-4d82-9b6d-9527f49fe7bb admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:29.620982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8302eb8-4642-40e0-9bfc-c18c2286fa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:29.622228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8302eb8-4642-40e0-9bfc-c18c2286fa6e admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:29.622515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8302eb8-4642-40e0-9bfc-c18c2286fa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:29.622789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8302eb8-4642-40e0-9bfc-c18c2286fa6e admin admin] PATH_INFO: `/v3/projects/4454dfb7be6541bfb5346fb4878661f2/users/411dc9875cfe41fe9d1e6d65665a4687/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:29.640606 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8302eb8-4642-40e0-9bfc-c18c2286fa6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4454dfb7be6541bfb5346fb4878661f2, user_id=411dc9875cfe41fe9d1e6d65665a4687, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:29.660932 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8302eb8-4642-40e0-9bfc-c18c2286fa6e admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:29.768622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 924/1834] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:29 2022] PUT /identity/v3/projects/4454dfb7be6541bfb5346fb4878661f2/users/411dc9875cfe41fe9d1e6d65665a4687/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:29.773585 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3b5b5-822c-4498-b2ef-151354806131 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:29.785661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3b5b5-822c-4498-b2ef-151354806131 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:29.854504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606f051-a1d3-4a2d-8784-be0b3858cfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:29.855110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606f051-a1d3-4a2d-8784-be0b3858cfa6 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:29.855390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606f051-a1d3-4a2d-8784-be0b3858cfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:29.856368 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606f051-a1d3-4a2d-8784-be0b3858cfa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:29.857329 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606f051-a1d3-4a2d-8784-be0b3858cfa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:29.858953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606f051-a1d3-4a2d-8784-be0b3858cfa6 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:29.868778 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 911/1835] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:29.873397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8302eb8-4642-40e0-9bfc-c18c2286fa6e admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:29.884867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8302eb8-4642-40e0-9bfc-c18c2286fa6e admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:29.890326 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea520d9-d01d-4fb1-a120-c814438f7a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:29.891146 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea520d9-d01d-4fb1-a120-c814438f7a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:29.891506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea520d9-d01d-4fb1-a120-c814438f7a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:29.895362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea520d9-d01d-4fb1-a120-c814438f7a1d admin admin] PATH_INFO: `/v3/projects/4454dfb7be6541bfb5346fb4878661f2/users/411dc9875cfe41fe9d1e6d65665a4687/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:29.899053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea520d9-d01d-4fb1-a120-c814438f7a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4454dfb7be6541bfb5346fb4878661f2, user_id=411dc9875cfe41fe9d1e6d65665a4687, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:29.900483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea520d9-d01d-4fb1-a120-c814438f7a1d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:29.912856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 925/1836] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:29 2022] PUT /identity/v3/projects/4454dfb7be6541bfb5346fb4878661f2/users/411dc9875cfe41fe9d1e6d65665a4687/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:29.924289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535fbc7-3644-4e3b-900e-561a13c09c60 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:29.924580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535fbc7-3644-4e3b-900e-561a13c09c60 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:29.924947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535fbc7-3644-4e3b-900e-561a13c09c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:29.952635 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-c535fbc7-3644-4e3b-900e-561a13c09c60 None None] MFA Rules not processed for user `411dc9875cfe41fe9d1e6d65665a4687`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:29.953856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c535fbc7-3644-4e3b-900e-561a13c09c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:30.030818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 912/1837] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:30.093683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea520d9-d01d-4fb1-a120-c814438f7a1d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:30.096973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5ea520d9-d01d-4fb1-a120-c814438f7a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:30.156727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea520d9-d01d-4fb1-a120-c814438f7a1d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:30.160412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f054e-def1-42d1-bde5-f4c47563bcfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:30.161069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f054e-def1-42d1-bde5-f4c47563bcfa service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:30.161383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f054e-def1-42d1-bde5-f4c47563bcfa service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:30.161658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f054e-def1-42d1-bde5-f4c47563bcfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:30.171297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f054e-def1-42d1-bde5-f4c47563bcfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:30.172687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f054e-def1-42d1-bde5-f4c47563bcfa service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:30.182639 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 926/1838] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:30.334161 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535fbc7-3644-4e3b-900e-561a13c09c60 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:30.348977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535fbc7-3644-4e3b-900e-561a13c09c60 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:30.359650 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0588d-554e-4a45-bff2-33544ee15f58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:30.360334 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0588d-554e-4a45-bff2-33544ee15f58 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:30.364777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0588d-554e-4a45-bff2-33544ee15f58 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:30.364777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0588d-554e-4a45-bff2-33544ee15f58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:30.441487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f054e-def1-42d1-bde5-f4c47563bcfa service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:30.475678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0588d-554e-4a45-bff2-33544ee15f58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:30.478490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0588d-554e-4a45-bff2-33544ee15f58 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:30.485955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:30.507792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 913/1839] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:30.585037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f054e-def1-42d1-bde5-f4c47563bcfa service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:30.591333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3783b-9e8d-45de-8840-1fb863c6f885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:30.593157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3783b-9e8d-45de-8840-1fb863c6f885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:30.593157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3783b-9e8d-45de-8840-1fb863c6f885 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:30.596502 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3783b-9e8d-45de-8840-1fb863c6f885 admin admin] PATH_INFO: `/v3/users/ed4c355e9a0147dcaf2eb359deb4d24b` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:30.597901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3783b-9e8d-45de-8840-1fb863c6f885 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed4c355e9a0147dcaf2eb359deb4d24b)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:30.602246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3783b-9e8d-45de-8840-1fb863c6f885 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:30.906053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:30.908201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5ef3783b-9e8d-45de-8840-1fb863c6f885 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed4c355e9a0147dcaf2eb359deb4d24b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:30.915520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5ef3783b-9e8d-45de-8840-1fb863c6f885 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed4c355e9a0147dcaf2eb359deb4d24b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:30.994399 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5ef3783b-9e8d-45de-8840-1fb863c6f885 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed4c355e9a0147dcaf2eb359deb4d24b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:30.995041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5ef3783b-9e8d-45de-8840-1fb863c6f885 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed4c355e9a0147dcaf2eb359deb4d24b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:31.010062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5ef3783b-9e8d-45de-8840-1fb863c6f885 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed4c355e9a0147dcaf2eb359deb4d24b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:31.017532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 927/1840] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:30 2022] DELETE /identity/v3/users/ed4c355e9a0147dcaf2eb359deb4d24b => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:31.021896 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9926f-9b6b-4655-89ad-42ac75e911c5 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:31.022350 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9926f-9b6b-4655-89ad-42ac75e911c5 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:31.022502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9926f-9b6b-4655-89ad-42ac75e911c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:31.042959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-66e9926f-9b6b-4655-89ad-42ac75e911c5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:31.045285 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-66e9926f-9b6b-4655-89ad-42ac75e911c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:31.112369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 914/1841] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:31.123435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3783b-9e8d-45de-8840-1fb863c6f885 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:31.124883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5ef3783b-9e8d-45de-8840-1fb863c6f885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:31.190154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3783b-9e8d-45de-8840-1fb863c6f885 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:31.194034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f05f8-d385-4255-89c3-8633e15c580b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:31.200472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f05f8-d385-4255-89c3-8633e15c580b service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:31.200472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f05f8-d385-4255-89c3-8633e15c580b service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:31.200472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f05f8-d385-4255-89c3-8633e15c580b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:31.206434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f05f8-d385-4255-89c3-8633e15c580b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:31.208084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f05f8-d385-4255-89c3-8633e15c580b service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:31.220343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 928/1842] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:31.688985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9926f-9b6b-4655-89ad-42ac75e911c5 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:31.698315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-66e9926f-9b6b-4655-89ad-42ac75e911c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:31.723265 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9926f-9b6b-4655-89ad-42ac75e911c5 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:31.736097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:31.737679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:31.738537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:31.738981 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] PATH_INFO: `/v3/projects/03351f5c52c24ad395c80329e5d640c0` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:31.744534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03351f5c52c24ad395c80329e5d640c0)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:31.747726 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:31.942139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03351f5c52c24ad395c80329e5d640c0', 'request_id': 'req-d3d0d429-453e-49b6-896e-fd1b313ab9c1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:31.948750 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03351f5c52c24ad395c80329e5d640c0', 'request_id': 'req-d3d0d429-453e-49b6-896e-fd1b313ab9c1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:32.074008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03351f5c52c24ad395c80329e5d640c0', 'request_id': 'req-d3d0d429-453e-49b6-896e-fd1b313ab9c1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:32.086027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] The token cache is being invalidate because project 03351f5c52c24ad395c80329e5d640c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:32.086325 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:32.087010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:32.088543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 915/1843] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:31 2022] DELETE /identity/v3/projects/03351f5c52c24ad395c80329e5d640c0 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:32.093358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f05f8-d385-4255-89c3-8633e15c580b service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:32.103074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f05f8-d385-4255-89c3-8633e15c580b service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:32.162492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fff480-0c53-48cc-8382-8d17101ddfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:32.164134 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fff480-0c53-48cc-8382-8d17101ddfec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:32.164571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fff480-0c53-48cc-8382-8d17101ddfec admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:32.165128 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fff480-0c53-48cc-8382-8d17101ddfec admin admin] PATH_INFO: `/v3/users/bad476bf094443ff80c79a74242adc00` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:32.166320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fff480-0c53-48cc-8382-8d17101ddfec admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad476bf094443ff80c79a74242adc00)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:32.169324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fff480-0c53-48cc-8382-8d17101ddfec admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:32.333519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-79fff480-0c53-48cc-8382-8d17101ddfec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad476bf094443ff80c79a74242adc00'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:32.340925 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-79fff480-0c53-48cc-8382-8d17101ddfec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad476bf094443ff80c79a74242adc00'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:32.398925 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-79fff480-0c53-48cc-8382-8d17101ddfec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad476bf094443ff80c79a74242adc00'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:32.399564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-79fff480-0c53-48cc-8382-8d17101ddfec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad476bf094443ff80c79a74242adc00'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:32.412823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-79fff480-0c53-48cc-8382-8d17101ddfec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad476bf094443ff80c79a74242adc00'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:32.419031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 929/1844] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:32 2022] DELETE /identity/v3/users/bad476bf094443ff80c79a74242adc00 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:33.162429 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:33.163960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:33.185063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0d429-453e-49b6-896e-fd1b313ab9c1 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:33.255318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fff480-0c53-48cc-8382-8d17101ddfec admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:33.257055 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-79fff480-0c53-48cc-8382-8d17101ddfec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:33.264311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a0c01-9625-4d37-b4d1-654c26f46f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:33.265059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a0c01-9625-4d37-b4d1-654c26f46f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:33.265331 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a0c01-9625-4d37-b4d1-654c26f46f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:33.265843 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a0c01-9625-4d37-b4d1-654c26f46f01 admin admin] PATH_INFO: `/v3/projects/4d17c11a2ae44d478cf677d73a99b2e8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:33.267467 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a0c01-9625-4d37-b4d1-654c26f46f01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d17c11a2ae44d478cf677d73a99b2e8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:33.269747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a0c01-9625-4d37-b4d1-654c26f46f01 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:33.290714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:33.336068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fff480-0c53-48cc-8382-8d17101ddfec admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:33.340985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf8bc3-32e8-4214-82f4-ec72547712dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:33.341634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8bc3-32e8-4214-82f4-ec72547712dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:33.341917 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8bc3-32e8-4214-82f4-ec72547712dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:33.342192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8bc3-32e8-4214-82f4-ec72547712dd admin admin] PATH_INFO: `/v3/users/3de732cc89c842298888d7cc2dea6449` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:33.343365 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf8bc3-32e8-4214-82f4-ec72547712dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3de732cc89c842298888d7cc2dea6449)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:33.345324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf8bc3-32e8-4214-82f4-ec72547712dd admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:33.401204 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-108a0c01-9625-4d37-b4d1-654c26f46f01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d17c11a2ae44d478cf677d73a99b2e8', 'request_id': 'req-108a0c01-9625-4d37-b4d1-654c26f46f01'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:33.408766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-108a0c01-9625-4d37-b4d1-654c26f46f01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d17c11a2ae44d478cf677d73a99b2e8', 'request_id': 'req-108a0c01-9625-4d37-b4d1-654c26f46f01'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:33.485519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-108a0c01-9625-4d37-b4d1-654c26f46f01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d17c11a2ae44d478cf677d73a99b2e8', 'request_id': 'req-108a0c01-9625-4d37-b4d1-654c26f46f01'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:33.497329 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-108a0c01-9625-4d37-b4d1-654c26f46f01 admin admin] The token cache is being invalidate because project 4d17c11a2ae44d478cf677d73a99b2e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:33.497638 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-108a0c01-9625-4d37-b4d1-654c26f46f01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:33.498282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-108a0c01-9625-4d37-b4d1-654c26f46f01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:33.499793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 916/1845] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:33 2022] DELETE /identity/v3/projects/4d17c11a2ae44d478cf677d73a99b2e8 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:33.504753 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764698c-ff2c-477c-bd10-c6f665d93dc1 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:33.505062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764698c-ff2c-477c-bd10-c6f665d93dc1 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:33.505332 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764698c-ff2c-477c-bd10-c6f665d93dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:33.525451 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-4764698c-ff2c-477c-bd10-c6f665d93dc1 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:33.527190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4764698c-ff2c-477c-bd10-c6f665d93dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:33.581243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7cdf8bc3-32e8-4214-82f4-ec72547712dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3de732cc89c842298888d7cc2dea6449'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:33.589080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7cdf8bc3-32e8-4214-82f4-ec72547712dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3de732cc89c842298888d7cc2dea6449'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:33.604823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 917/1846] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:33.612948 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ab3bf-9617-48ae-85c5-d9cfcdcd46bd None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:33.613236 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ab3bf-9617-48ae-85c5-d9cfcdcd46bd None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:33.613511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ab3bf-9617-48ae-85c5-d9cfcdcd46bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:33.633502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-493ab3bf-9617-48ae-85c5-d9cfcdcd46bd None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:33.635779 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-493ab3bf-9617-48ae-85c5-d9cfcdcd46bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:33.643775 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 918/1847] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:33.647567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ab3bf-9617-48ae-85c5-d9cfcdcd46bd None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:33.647996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7cdf8bc3-32e8-4214-82f4-ec72547712dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3de732cc89c842298888d7cc2dea6449'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:33.648839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7cdf8bc3-32e8-4214-82f4-ec72547712dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3de732cc89c842298888d7cc2dea6449'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:33.656996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ab3bf-9617-48ae-85c5-d9cfcdcd46bd None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:33.662239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7cdf8bc3-32e8-4214-82f4-ec72547712dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3de732cc89c842298888d7cc2dea6449'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:33.663290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db2991-d44b-4db2-8a04-32038970bb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:33.663908 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db2991-d44b-4db2-8a04-32038970bb38 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:33.664193 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db2991-d44b-4db2-8a04-32038970bb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:33.664471 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db2991-d44b-4db2-8a04-32038970bb38 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:33.665163 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db2991-d44b-4db2-8a04-32038970bb38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:33.665472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db2991-d44b-4db2-8a04-32038970bb38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:33.667147 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db2991-d44b-4db2-8a04-32038970bb38 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:33.670168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 930/1848] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:33 2022] DELETE /identity/v3/users/3de732cc89c842298888d7cc2dea6449 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:33.675181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192a956-21ba-448d-8017-763cae1c91f9 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:33.675539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192a956-21ba-448d-8017-763cae1c91f9 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:33.675817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192a956-21ba-448d-8017-763cae1c91f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:33.682953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 919/1849] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:52:33.685854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db2991-d44b-4db2-8a04-32038970bb38 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:33.688222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-77db2991-d44b-4db2-8a04-32038970bb38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:33.696799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-7192a956-21ba-448d-8017-763cae1c91f9 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:33.698330 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7192a956-21ba-448d-8017-763cae1c91f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:33.702955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db2991-d44b-4db2-8a04-32038970bb38 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:33.757124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 931/1850] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:33.760489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa246b1-9cef-4c65-8493-a9e9fe552193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:33.761291 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa246b1-9cef-4c65-8493-a9e9fe552193 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:33.761634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa246b1-9cef-4c65-8493-a9e9fe552193 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:33.762613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa246b1-9cef-4c65-8493-a9e9fe552193 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:33.763338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa246b1-9cef-4c65-8493-a9e9fe552193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:33.767337 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa246b1-9cef-4c65-8493-a9e9fe552193 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:33.774960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192a956-21ba-448d-8017-763cae1c91f9 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:33.776516 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7192a956-21ba-448d-8017-763cae1c91f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:33.835384 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192a956-21ba-448d-8017-763cae1c91f9 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:33.839111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a117b-dc99-46d3-9a15-09fa24eb5b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:33.842005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a117b-dc99-46d3-9a15-09fa24eb5b69 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:33.842005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a117b-dc99-46d3-9a15-09fa24eb5b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:33.842005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a117b-dc99-46d3-9a15-09fa24eb5b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:33.849407 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a117b-dc99-46d3-9a15-09fa24eb5b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:33.850974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a117b-dc99-46d3-9a15-09fa24eb5b69 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:33.861907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 932/1851] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:33.898298 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 920/1852] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:33 2022] POST /identity/v3/projects => generated 387 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:33.899210 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a117b-dc99-46d3-9a15-09fa24eb5b69 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:33.920775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a117b-dc99-46d3-9a15-09fa24eb5b69 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:33.999384 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373f36d-f8e8-45e6-a178-e030665d747e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:34.000963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373f36d-f8e8-45e6-a178-e030665d747e admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:34.002333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373f36d-f8e8-45e6-a178-e030665d747e admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:34.002333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373f36d-f8e8-45e6-a178-e030665d747e admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:34.002333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373f36d-f8e8-45e6-a178-e030665d747e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:34.004677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373f36d-f8e8-45e6-a178-e030665d747e admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:34.121233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 933/1853] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:33 2022] POST /identity/v3/users => generated 294 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:34.126713 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa246b1-9cef-4c65-8493-a9e9fe552193 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:34.141511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa246b1-9cef-4c65-8493-a9e9fe552193 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:34.162725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc45b2-5d6a-4c10-a815-115b48e53565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:34.163867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc45b2-5d6a-4c10-a815-115b48e53565 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:34.164152 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc45b2-5d6a-4c10-a815-115b48e53565 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:34.164427 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc45b2-5d6a-4c10-a815-115b48e53565 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:34.165112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc45b2-5d6a-4c10-a815-115b48e53565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:34.167533 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc45b2-5d6a-4c10-a815-115b48e53565 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:34.185372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 921/1854] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:34.189414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373f36d-f8e8-45e6-a178-e030665d747e admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:34.201510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373f36d-f8e8-45e6-a178-e030665d747e admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:34.207536 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd1c30-1b46-4849-8fff-f4a5fd383c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:34.208543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd1c30-1b46-4849-8fff-f4a5fd383c67 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:34.208947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd1c30-1b46-4849-8fff-f4a5fd383c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:34.209230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd1c30-1b46-4849-8fff-f4a5fd383c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:34.210081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd1c30-1b46-4849-8fff-f4a5fd383c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:34.211885 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd1c30-1b46-4849-8fff-f4a5fd383c67 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:34.226581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 934/1855] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:34.232256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc45b2-5d6a-4c10-a815-115b48e53565 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:34.251703 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc45b2-5d6a-4c10-a815-115b48e53565 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:34.257396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2a882-d057-434d-a3f9-d996d475eef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:34.258220 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2a882-d057-434d-a3f9-d996d475eef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:34.258503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2a882-d057-434d-a3f9-d996d475eef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:34.258913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2a882-d057-434d-a3f9-d996d475eef5 admin admin] PATH_INFO: `/v3/projects/c8d63332443c4bfab79987450d4a8176/users/8d92b28d08f74a27a66fec6b320460dc/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:34.276425 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2a882-d057-434d-a3f9-d996d475eef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8d63332443c4bfab79987450d4a8176, user_id=8d92b28d08f74a27a66fec6b320460dc, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:34.279784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2a882-d057-434d-a3f9-d996d475eef5 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:34.421601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 922/1856] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:34 2022] PUT /identity/v3/projects/c8d63332443c4bfab79987450d4a8176/users/8d92b28d08f74a27a66fec6b320460dc/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:34.434796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82a866-c230-4dcc-8e37-8e52d55cd8cc None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:34.435426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82a866-c230-4dcc-8e37-8e52d55cd8cc None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:34.435523 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82a866-c230-4dcc-8e37-8e52d55cd8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:34.466480 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-ce82a866-c230-4dcc-8e37-8e52d55cd8cc None None] MFA Rules not processed for user `8d92b28d08f74a27a66fec6b320460dc`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:34.468731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ce82a866-c230-4dcc-8e37-8e52d55cd8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:34.565033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 935/1857] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:34.573498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2a882-d057-434d-a3f9-d996d475eef5 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:34.652602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2a882-d057-434d-a3f9-d996d475eef5 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:34.660356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9f676-22be-4667-8479-8b3d558e5b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:34.661051 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9f676-22be-4667-8479-8b3d558e5b2a admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:34.661331 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9f676-22be-4667-8479-8b3d558e5b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:34.661741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9f676-22be-4667-8479-8b3d558e5b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:34.662404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9f676-22be-4667-8479-8b3d558e5b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:34.664387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9f676-22be-4667-8479-8b3d558e5b2a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:34.820697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 923/1858] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:34 2022] POST /identity/v3/projects => generated 387 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:34.823087 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82a866-c230-4dcc-8e37-8e52d55cd8cc None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:34.834701 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82a866-c230-4dcc-8e37-8e52d55cd8cc None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:34.911795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bcffc-5e1f-4e8b-92dc-c7556d679486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:34.912441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bcffc-5e1f-4e8b-92dc-c7556d679486 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:34.912739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bcffc-5e1f-4e8b-92dc-c7556d679486 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:34.913017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bcffc-5e1f-4e8b-92dc-c7556d679486 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:34.913660 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bcffc-5e1f-4e8b-92dc-c7556d679486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:34.915864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bcffc-5e1f-4e8b-92dc-c7556d679486 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:35.043068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 936/1859] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:34 2022] POST /identity/v3/users => generated 300 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:35.044951 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9f676-22be-4667-8479-8b3d558e5b2a admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:35.055046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9f676-22be-4667-8479-8b3d558e5b2a admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:35.059962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332d4b5-72d0-4149-aac4-68ee4aa7f58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:35.067387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332d4b5-72d0-4149-aac4-68ee4aa7f58c admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:35.067387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332d4b5-72d0-4149-aac4-68ee4aa7f58c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:35.067387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332d4b5-72d0-4149-aac4-68ee4aa7f58c admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:35.067387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332d4b5-72d0-4149-aac4-68ee4aa7f58c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:35.067387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332d4b5-72d0-4149-aac4-68ee4aa7f58c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:35.075328 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 924/1860] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:35.079908 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bcffc-5e1f-4e8b-92dc-c7556d679486 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:35.090501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bcffc-5e1f-4e8b-92dc-c7556d679486 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:35.095091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b9cd7-e8a6-4b61-ac15-da1b39f34e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:35.101367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b9cd7-e8a6-4b61-ac15-da1b39f34e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:35.101367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b9cd7-e8a6-4b61-ac15-da1b39f34e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:35.101367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b9cd7-e8a6-4b61-ac15-da1b39f34e47 admin admin] PATH_INFO: `/v3/projects/5e8fab1bef4b41bbbdd2911897b7e904/users/0d148ea348514447bf6d5257bc1a61ee/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:35.112496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1462ce-9a0d-4ad7-bada-90623087f61d None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:35.112878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1462ce-9a0d-4ad7-bada-90623087f61d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:35.113118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1462ce-9a0d-4ad7-bada-90623087f61d None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:35.114648 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 925/1861] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 24 07:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:52:35.124841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1462ce-9a0d-4ad7-bada-90623087f61d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:35.125683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:35.126155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0c1462ce-9a0d-4ad7-bada-90623087f61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:35.130816 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b9cd7-e8a6-4b61-ac15-da1b39f34e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e8fab1bef4b41bbbdd2911897b7e904, user_id=0d148ea348514447bf6d5257bc1a61ee, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:35.139141 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b9cd7-e8a6-4b61-ac15-da1b39f34e47 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:35.154837 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1462ce-9a0d-4ad7-bada-90623087f61d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:35.160026 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:35.160678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:35.160963 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:35.161234 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] PATH_INFO: `/v3/projects/ed979d81272f4041af5178fbe7b78cf6` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:35.162378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed979d81272f4041af5178fbe7b78cf6)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:35.165142 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:35.257641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 937/1862] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:35 2022] PUT /identity/v3/projects/5e8fab1bef4b41bbbdd2911897b7e904/users/0d148ea348514447bf6d5257bc1a61ee/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:35.261905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977056a-b38c-42d2-baba-30095c627ecd None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:35.262187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977056a-b38c-42d2-baba-30095c627ecd None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:35.262465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977056a-b38c-42d2-baba-30095c627ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:35.274063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed979d81272f4041af5178fbe7b78cf6', 'request_id': 'req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.283741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed979d81272f4041af5178fbe7b78cf6', 'request_id': 'req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.287487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-b977056a-b38c-42d2-baba-30095c627ecd None None] MFA Rules not processed for user `bdf81f09006a42299c0807f77f16c748`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:35.288757 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b977056a-b38c-42d2-baba-30095c627ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:35.359674 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed979d81272f4041af5178fbe7b78cf6', 'request_id': 'req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.376073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] The token cache is being invalidate because project ed979d81272f4041af5178fbe7b78cf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:35.376783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.377620 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.379692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 926/1863] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:35 2022] DELETE /identity/v3/projects/ed979d81272f4041af5178fbe7b78cf6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:35.380693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:35.382233 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:35.385167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 938/1864] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 24 07:52:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:35.386082 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977056a-b38c-42d2-baba-30095c627ecd None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:35.388214 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b977056a-b38c-42d2-baba-30095c627ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:35.469370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977056a-b38c-42d2-baba-30095c627ecd None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:35.474668 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d878d-8779-4a47-8cc9-ee6bbeda014d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:35.475391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d878d-8779-4a47-8cc9-ee6bbeda014d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:35.475842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d878d-8779-4a47-8cc9-ee6bbeda014d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:35.476217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d878d-8779-4a47-8cc9-ee6bbeda014d admin admin] PATH_INFO: `/v3/users/f45dd4990fd24efe99ec730ca22fc7d6` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:35.477610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d878d-8779-4a47-8cc9-ee6bbeda014d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f45dd4990fd24efe99ec730ca22fc7d6)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:35.479934 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d878d-8779-4a47-8cc9-ee6bbeda014d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:35.485219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676dbb0-e1db-49a8-a2c6-44494d1fe51c admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:35.490282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9f7af-7a97-4f71-88c6-fb9f290d903b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:35.497052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9f7af-7a97-4f71-88c6-fb9f290d903b admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:35.497052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9f7af-7a97-4f71-88c6-fb9f290d903b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:35.497052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9f7af-7a97-4f71-88c6-fb9f290d903b admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:35.497052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9f7af-7a97-4f71-88c6-fb9f290d903b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:35.497052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9f7af-7a97-4f71-88c6-fb9f290d903b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:35.505912 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 927/1865] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:35.506983 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9f7af-7a97-4f71-88c6-fb9f290d903b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:35.511897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f9a9f7af-7a97-4f71-88c6-fb9f290d903b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:35.525399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9f7af-7a97-4f71-88c6-fb9f290d903b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:35.531471 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d5ca3-e795-43c3-9d80-35856924409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:35.531471 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d5ca3-e795-43c3-9d80-35856924409d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:35.531868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d5ca3-e795-43c3-9d80-35856924409d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:35.531868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d5ca3-e795-43c3-9d80-35856924409d admin admin] PATH_INFO: `/v3/users/04bdfda2dede483498f42ee7dfe288d2` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:35.533054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d5ca3-e795-43c3-9d80-35856924409d admin admin] RBAC: Authorizing `identity:delete_user(user_id=04bdfda2dede483498f42ee7dfe288d2)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:35.535057 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d5ca3-e795-43c3-9d80-35856924409d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:35.753975 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b97d878d-8779-4a47-8cc9-ee6bbeda014d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f45dd4990fd24efe99ec730ca22fc7d6'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.761463 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b97d878d-8779-4a47-8cc9-ee6bbeda014d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f45dd4990fd24efe99ec730ca22fc7d6'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.804498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-869d5ca3-e795-43c3-9d80-35856924409d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04bdfda2dede483498f42ee7dfe288d2'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.874405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b97d878d-8779-4a47-8cc9-ee6bbeda014d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f45dd4990fd24efe99ec730ca22fc7d6'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.875121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b97d878d-8779-4a47-8cc9-ee6bbeda014d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f45dd4990fd24efe99ec730ca22fc7d6'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.890191 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b97d878d-8779-4a47-8cc9-ee6bbeda014d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f45dd4990fd24efe99ec730ca22fc7d6'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.897381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 939/1866] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:35 2022] DELETE /identity/v3/users/f45dd4990fd24efe99ec730ca22fc7d6 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:35.898358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d878d-8779-4a47-8cc9-ee6bbeda014d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:35.899872 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b97d878d-8779-4a47-8cc9-ee6bbeda014d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:35.942143 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-869d5ca3-e795-43c3-9d80-35856924409d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04bdfda2dede483498f42ee7dfe288d2'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.942852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-869d5ca3-e795-43c3-9d80-35856924409d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04bdfda2dede483498f42ee7dfe288d2'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.952968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-869d5ca3-e795-43c3-9d80-35856924409d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04bdfda2dede483498f42ee7dfe288d2'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.970322 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-869d5ca3-e795-43c3-9d80-35856924409d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04bdfda2dede483498f42ee7dfe288d2'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:35.982496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 928/1867] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:35 2022] DELETE /identity/v3/users/04bdfda2dede483498f42ee7dfe288d2 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:35.982496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d5ca3-e795-43c3-9d80-35856924409d admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:35.984134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-869d5ca3-e795-43c3-9d80-35856924409d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:36.003228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d878d-8779-4a47-8cc9-ee6bbeda014d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:36.085253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c950be4-ef26-48bf-b6a3-56ed48ba1968 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:36.086963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c950be4-ef26-48bf-b6a3-56ed48ba1968 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:36.087314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c950be4-ef26-48bf-b6a3-56ed48ba1968 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:36.087878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c950be4-ef26-48bf-b6a3-56ed48ba1968 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:36.090046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2c950be4-ef26-48bf-b6a3-56ed48ba1968 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:36.097837 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d5ca3-e795-43c3-9d80-35856924409d admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:36.105284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbee29-ff73-4c63-a590-a00261a7da35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:36.106924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbee29-ff73-4c63-a590-a00261a7da35 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:36.107359 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbee29-ff73-4c63-a590-a00261a7da35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:36.107833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbee29-ff73-4c63-a590-a00261a7da35 admin admin] PATH_INFO: `/v3/projects/5e8fab1bef4b41bbbdd2911897b7e904/users/0d148ea348514447bf6d5257bc1a61ee/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:36.111108 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbee29-ff73-4c63-a590-a00261a7da35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e8fab1bef4b41bbbdd2911897b7e904, user_id=0d148ea348514447bf6d5257bc1a61ee, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:36.116166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbee29-ff73-4c63-a590-a00261a7da35 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:36.130127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c950be4-ef26-48bf-b6a3-56ed48ba1968 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:36.136068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c950be4-ef26-48bf-b6a3-56ed48ba1968 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:36.142486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 929/1868] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:35 2022] PUT /identity/v3/projects/5e8fab1bef4b41bbbdd2911897b7e904/users/0d148ea348514447bf6d5257bc1a61ee/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:36.146991 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73177-4874-4926-872f-dba4be6af4ea None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:36.147501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73177-4874-4926-872f-dba4be6af4ea None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:36.148454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73177-4874-4926-872f-dba4be6af4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:36.174534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 940/1869] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:36.176573 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-eda73177-4874-4926-872f-dba4be6af4ea None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:36.178478 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-eda73177-4874-4926-872f-dba4be6af4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:36.181465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a0ef8-059e-4a9e-a346-4109ab1647cc None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:36.182153 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a0ef8-059e-4a9e-a346-4109ab1647cc None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:36.185984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a0ef8-059e-4a9e-a346-4109ab1647cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:36.260960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 930/1870] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:36.269058 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:36.278513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-898a0ef8-059e-4a9e-a346-4109ab1647cc None None] MFA Rules not processed for user `0d148ea348514447bf6d5257bc1a61ee`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:36.282229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-898a0ef8-059e-4a9e-a346-4109ab1647cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:36.297817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda73177-4874-4926-872f-dba4be6af4ea None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:36.304219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-eda73177-4874-4926-872f-dba4be6af4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:36.404818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda73177-4874-4926-872f-dba4be6af4ea None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:36.416612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1475ed-57e9-466e-b75f-55f65320fa9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:36.418824 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1475ed-57e9-466e-b75f-55f65320fa9b service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:36.421988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1475ed-57e9-466e-b75f-55f65320fa9b service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:36.422551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1475ed-57e9-466e-b75f-55f65320fa9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:36.438679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 941/1871] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:36.441233 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1475ed-57e9-466e-b75f-55f65320fa9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:36.443450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1475ed-57e9-466e-b75f-55f65320fa9b service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:36.448866 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a0ef8-059e-4a9e-a346-4109ab1647cc None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:36.450914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-898a0ef8-059e-4a9e-a346-4109ab1647cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:36.479115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 931/1872] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:36.543123 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a0ef8-059e-4a9e-a346-4109ab1647cc None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:36.551165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4429088-ee1d-4480-a389-cbdf0f0a8e6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:36.552350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4429088-ee1d-4480-a389-cbdf0f0a8e6b service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:36.552844 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4429088-ee1d-4480-a389-cbdf0f0a8e6b service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:36.553281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4429088-ee1d-4480-a389-cbdf0f0a8e6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:36.568929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4429088-ee1d-4480-a389-cbdf0f0a8e6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:36.570907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4429088-ee1d-4480-a389-cbdf0f0a8e6b service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:36.585161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 942/1873] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:36.647772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1475ed-57e9-466e-b75f-55f65320fa9b service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:36.672034 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1475ed-57e9-466e-b75f-55f65320fa9b service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:36.678707 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de662740-1b30-48a7-9cfe-9e77f3cbf8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:36.679426 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de662740-1b30-48a7-9cfe-9e77f3cbf8c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:36.679782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de662740-1b30-48a7-9cfe-9e77f3cbf8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:36.680134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de662740-1b30-48a7-9cfe-9e77f3cbf8c3 admin admin] PATH_INFO: `/v3/users/8d92b28d08f74a27a66fec6b320460dc` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:36.681657 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de662740-1b30-48a7-9cfe-9e77f3cbf8c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d92b28d08f74a27a66fec6b320460dc)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:36.685029 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de662740-1b30-48a7-9cfe-9e77f3cbf8c3 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:36.875856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-de662740-1b30-48a7-9cfe-9e77f3cbf8c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d92b28d08f74a27a66fec6b320460dc'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:36.961186 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-de662740-1b30-48a7-9cfe-9e77f3cbf8c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d92b28d08f74a27a66fec6b320460dc'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:36.978414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-de662740-1b30-48a7-9cfe-9e77f3cbf8c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d92b28d08f74a27a66fec6b320460dc'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:37.001232 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-de662740-1b30-48a7-9cfe-9e77f3cbf8c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d92b28d08f74a27a66fec6b320460dc'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:37.035697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-de662740-1b30-48a7-9cfe-9e77f3cbf8c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d92b28d08f74a27a66fec6b320460dc'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:37.049008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 932/1874] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:36 2022] DELETE /identity/v3/users/8d92b28d08f74a27a66fec6b320460dc => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:37.053776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203399a4-0661-41cc-96d4-13788174f98a None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:37.054068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203399a4-0661-41cc-96d4-13788174f98a None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:37.054351 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203399a4-0661-41cc-96d4-13788174f98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:37.084132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-203399a4-0661-41cc-96d4-13788174f98a None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:37.085698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-203399a4-0661-41cc-96d4-13788174f98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:37.210402 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 943/1875] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:37.240614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de662740-1b30-48a7-9cfe-9e77f3cbf8c3 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:37.244937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-de662740-1b30-48a7-9cfe-9e77f3cbf8c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:37.304183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203399a4-0661-41cc-96d4-13788174f98a None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:37.305225 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-203399a4-0661-41cc-96d4-13788174f98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:37.329836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203399a4-0661-41cc-96d4-13788174f98a None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:37.344399 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0013072-2259-4342-a184-1b8bf76424c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:37.345791 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0013072-2259-4342-a184-1b8bf76424c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:37.345941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0013072-2259-4342-a184-1b8bf76424c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:37.346676 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0013072-2259-4342-a184-1b8bf76424c8 admin admin] PATH_INFO: `/v3/projects/db9a8da245764743a8288c67d12a64d8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:37.351481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0013072-2259-4342-a184-1b8bf76424c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db9a8da245764743a8288c67d12a64d8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:37.359031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0013072-2259-4342-a184-1b8bf76424c8 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:37.368294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de662740-1b30-48a7-9cfe-9e77f3cbf8c3 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:37.377100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:37.379335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3908d1-4d40-43fb-9eb2-4b354e7b36cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:37.380987 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3908d1-4d40-43fb-9eb2-4b354e7b36cc service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:37.381444 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3908d1-4d40-43fb-9eb2-4b354e7b36cc service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:37.381957 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3908d1-4d40-43fb-9eb2-4b354e7b36cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:37.400084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3908d1-4d40-43fb-9eb2-4b354e7b36cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:37.400084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3908d1-4d40-43fb-9eb2-4b354e7b36cc service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:37.416034 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 933/1876] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:37.540832 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-c0013072-2259-4342-a184-1b8bf76424c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db9a8da245764743a8288c67d12a64d8', 'request_id': 'req-c0013072-2259-4342-a184-1b8bf76424c8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:37.549009 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-c0013072-2259-4342-a184-1b8bf76424c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db9a8da245764743a8288c67d12a64d8', 'request_id': 'req-c0013072-2259-4342-a184-1b8bf76424c8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:37.558459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-c0013072-2259-4342-a184-1b8bf76424c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db9a8da245764743a8288c67d12a64d8', 'request_id': 'req-c0013072-2259-4342-a184-1b8bf76424c8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:37.599490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3908d1-4d40-43fb-9eb2-4b354e7b36cc service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:37.601066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ee3908d1-4d40-43fb-9eb2-4b354e7b36cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:37.614473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3908d1-4d40-43fb-9eb2-4b354e7b36cc service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:37.667084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-c0013072-2259-4342-a184-1b8bf76424c8 admin admin] The token cache is being invalidate because project db9a8da245764743a8288c67d12a64d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:37.667772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-c0013072-2259-4342-a184-1b8bf76424c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:37.669397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-c0013072-2259-4342-a184-1b8bf76424c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:37.671126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 944/1877] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:37 2022] DELETE /identity/v3/projects/db9a8da245764743a8288c67d12a64d8 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:37.675904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504b56f-4652-480b-8a7c-15379d32e2a1 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:37.676203 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504b56f-4652-480b-8a7c-15379d32e2a1 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:37.676488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504b56f-4652-480b-8a7c-15379d32e2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:37.692942 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:37.693920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:37.694390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:37.695036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] PATH_INFO: `/v3/projects/3159f0caee954ba0bc7decc62df56123` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:37.696715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3159f0caee954ba0bc7decc62df56123)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:37.698846 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:37.713962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-7504b56f-4652-480b-8a7c-15379d32e2a1 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:37.713962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7504b56f-4652-480b-8a7c-15379d32e2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:37.725598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 945/1878] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:37.727868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11c034-0a0f-4b28-9b67-580825a8d4f9 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:37.728217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11c034-0a0f-4b28-9b67-580825a8d4f9 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:37.729171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11c034-0a0f-4b28-9b67-580825a8d4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:37.751959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-8a11c034-0a0f-4b28-9b67-580825a8d4f9 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:37.753286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8a11c034-0a0f-4b28-9b67-580825a8d4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:37.762457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 946/1879] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:37.766451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11c034-0a0f-4b28-9b67-580825a8d4f9 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:37.779969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11c034-0a0f-4b28-9b67-580825a8d4f9 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:37.789563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2a330-5a19-4526-99e6-89f5eadaca76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:37.796564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2a330-5a19-4526-99e6-89f5eadaca76 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:37.796564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2a330-5a19-4526-99e6-89f5eadaca76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:37.796564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2a330-5a19-4526-99e6-89f5eadaca76 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:37.796564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2a330-5a19-4526-99e6-89f5eadaca76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:37.796564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2a330-5a19-4526-99e6-89f5eadaca76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:37.796564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2a330-5a19-4526-99e6-89f5eadaca76 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:37.833450 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 947/1880] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:52:37.836035 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2a330-5a19-4526-99e6-89f5eadaca76 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:37.847312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2a330-5a19-4526-99e6-89f5eadaca76 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:37.856490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3159f0caee954ba0bc7decc62df56123', 'request_id': 'req-fac17615-3434-40c2-a67e-ae4bc3440144'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:37.866356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3159f0caee954ba0bc7decc62df56123', 'request_id': 'req-fac17615-3434-40c2-a67e-ae4bc3440144'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:37.914684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731d760-26bb-4d37-b125-cc72d303a61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:37.915298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731d760-26bb-4d37-b125-cc72d303a61c admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:37.915578 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731d760-26bb-4d37-b125-cc72d303a61c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:37.916220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731d760-26bb-4d37-b125-cc72d303a61c admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:37.916597 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731d760-26bb-4d37-b125-cc72d303a61c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:37.919228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731d760-26bb-4d37-b125-cc72d303a61c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:37.948567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3159f0caee954ba0bc7decc62df56123', 'request_id': 'req-fac17615-3434-40c2-a67e-ae4bc3440144'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:37.962261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] The token cache is being invalidate because project 3159f0caee954ba0bc7decc62df56123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:37.962802 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:37.963864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:37.969642 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 934/1881] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:37 2022] DELETE /identity/v3/projects/3159f0caee954ba0bc7decc62df56123 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:37.970530 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:37.971997 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:37.988000 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac17615-3434-40c2-a67e-ae4bc3440144 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:37.993938 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80928a8a-0299-4ba4-ad82-635ecc3fa7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:37.995529 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80928a8a-0299-4ba4-ad82-635ecc3fa7b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:37.995888 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80928a8a-0299-4ba4-ad82-635ecc3fa7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:37.998967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80928a8a-0299-4ba4-ad82-635ecc3fa7b8 admin admin] PATH_INFO: `/v3/users/0432d7e636ee41edae2d1a1bff0bc690` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:37.999857 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80928a8a-0299-4ba4-ad82-635ecc3fa7b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0432d7e636ee41edae2d1a1bff0bc690)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:38.007839 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80928a8a-0299-4ba4-ad82-635ecc3fa7b8 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:38.883180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 948/1882] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:37 2022] POST /identity/v3/projects => generated 381 bytes in 1042 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:52:38.886412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731d760-26bb-4d37-b125-cc72d303a61c admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:38.888331 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e731d760-26bb-4d37-b125-cc72d303a61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:38.951905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731d760-26bb-4d37-b125-cc72d303a61c admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:38.957636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65315a5-5bd7-4aeb-a88b-525880588097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:38.958386 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65315a5-5bd7-4aeb-a88b-525880588097 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:38.958708 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65315a5-5bd7-4aeb-a88b-525880588097 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:38.959046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65315a5-5bd7-4aeb-a88b-525880588097 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:38.959785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65315a5-5bd7-4aeb-a88b-525880588097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:38.962012 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65315a5-5bd7-4aeb-a88b-525880588097 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:39.260829 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 949/1883] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:38 2022] POST /identity/v3/users => generated 291 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:52:39.260829 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65315a5-5bd7-4aeb-a88b-525880588097 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:39.272491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-80928a8a-0299-4ba4-ad82-635ecc3fa7b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0432d7e636ee41edae2d1a1bff0bc690'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:39.275818 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e65315a5-5bd7-4aeb-a88b-525880588097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:39.292192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65315a5-5bd7-4aeb-a88b-525880588097 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:39.298373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:39.299498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:39.299498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:39.299784 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] PATH_INFO: `/v3/projects/c8d63332443c4bfab79987450d4a8176` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:39.301184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8d63332443c4bfab79987450d4a8176)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:39.306699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:39.434705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-80928a8a-0299-4ba4-ad82-635ecc3fa7b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0432d7e636ee41edae2d1a1bff0bc690'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:39.435636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-80928a8a-0299-4ba4-ad82-635ecc3fa7b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0432d7e636ee41edae2d1a1bff0bc690'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:39.452889 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-80928a8a-0299-4ba4-ad82-635ecc3fa7b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0432d7e636ee41edae2d1a1bff0bc690'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:39.462428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8d63332443c4bfab79987450d4a8176', 'request_id': 'req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:39.464677 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-80928a8a-0299-4ba4-ad82-635ecc3fa7b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0432d7e636ee41edae2d1a1bff0bc690'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:39.475387 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8d63332443c4bfab79987450d4a8176', 'request_id': 'req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:39.475586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 935/1884] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:37 2022] DELETE /identity/v3/users/0432d7e636ee41edae2d1a1bff0bc690 => generated 0 bytes in 1505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:39.476796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80928a8a-0299-4ba4-ad82-635ecc3fa7b8 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:39.481442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-80928a8a-0299-4ba4-ad82-635ecc3fa7b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:39.488059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8d63332443c4bfab79987450d4a8176', 'request_id': 'req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:39.512836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:39.572193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] The token cache is being invalidate because project c8d63332443c4bfab79987450d4a8176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:39.572763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:39.579234 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:39.579234 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 950/1885] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:39 2022] DELETE /identity/v3/projects/c8d63332443c4bfab79987450d4a8176 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:39.581086 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:39.582570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:39.598218 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80928a8a-0299-4ba4-ad82-635ecc3fa7b8 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:39.600307 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04fbb6-9399-4865-ad9d-e460ee306128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:39.600938 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04fbb6-9399-4865-ad9d-e460ee306128 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:39.601276 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04fbb6-9399-4865-ad9d-e460ee306128 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:39.601552 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04fbb6-9399-4865-ad9d-e460ee306128 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:39.602891 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04fbb6-9399-4865-ad9d-e460ee306128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:39.605081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04fbb6-9399-4865-ad9d-e460ee306128 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:39.619404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 936/1886] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:39.620776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04fbb6-9399-4865-ad9d-e460ee306128 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:39.626756 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9e04fbb6-9399-4865-ad9d-e460ee306128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:39.670542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ad7ec-d6eb-4ed5-b91c-74833c0d8243 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:39.677433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd88e2-3463-4003-8c7d-3d0657e3371e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:39.678474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04fbb6-9399-4865-ad9d-e460ee306128 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:39.678617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd88e2-3463-4003-8c7d-3d0657e3371e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:39.679220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd88e2-3463-4003-8c7d-3d0657e3371e admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:39.679570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd88e2-3463-4003-8c7d-3d0657e3371e admin admin] PATH_INFO: `/v3/users/0d148ea348514447bf6d5257bc1a61ee` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:39.680932 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd88e2-3463-4003-8c7d-3d0657e3371e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d148ea348514447bf6d5257bc1a61ee)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:39.684275 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd88e2-3463-4003-8c7d-3d0657e3371e admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:39.703500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4624d-98a4-4527-acc0-76dd73db3c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:39.705335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4624d-98a4-4527-acc0-76dd73db3c48 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:39.705955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4624d-98a4-4527-acc0-76dd73db3c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:39.707041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4624d-98a4-4527-acc0-76dd73db3c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:39.709742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4624d-98a4-4527-acc0-76dd73db3c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:39.711556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4624d-98a4-4527-acc0-76dd73db3c48 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:39.747376 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 937/1887] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:39.749998 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4624d-98a4-4527-acc0-76dd73db3c48 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:39.766604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4624d-98a4-4527-acc0-76dd73db3c48 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:39.772469 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467601d-3e42-490b-b7e9-4bd2b3ca3b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:39.774609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467601d-3e42-490b-b7e9-4bd2b3ca3b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:39.774609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467601d-3e42-490b-b7e9-4bd2b3ca3b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:39.774609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467601d-3e42-490b-b7e9-4bd2b3ca3b83 admin admin] PATH_INFO: `/v3/projects/5c9a7f07d478479fa050eee69bc79516/users/3ddd62d59df24013b92fa85a51718d57/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:39.794047 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467601d-3e42-490b-b7e9-4bd2b3ca3b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c9a7f07d478479fa050eee69bc79516, user_id=3ddd62d59df24013b92fa85a51718d57, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:39.797591 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467601d-3e42-490b-b7e9-4bd2b3ca3b83 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:39.962816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 938/1888] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:39 2022] PUT /identity/v3/projects/5c9a7f07d478479fa050eee69bc79516/users/3ddd62d59df24013b92fa85a51718d57/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:39.973334 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76198117-83c3-408b-bd0a-444071dfe8da None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:39.973724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76198117-83c3-408b-bd0a-444071dfe8da None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:39.974102 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76198117-83c3-408b-bd0a-444071dfe8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:39.989165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3bdd88e2-3463-4003-8c7d-3d0657e3371e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d148ea348514447bf6d5257bc1a61ee'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:40.003667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3bdd88e2-3463-4003-8c7d-3d0657e3371e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d148ea348514447bf6d5257bc1a61ee'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:40.021344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-76198117-83c3-408b-bd0a-444071dfe8da None None] MFA Rules not processed for user `3ddd62d59df24013b92fa85a51718d57`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:40.022924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-76198117-83c3-408b-bd0a-444071dfe8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:40.050650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3bdd88e2-3463-4003-8c7d-3d0657e3371e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d148ea348514447bf6d5257bc1a61ee'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:40.056519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3bdd88e2-3463-4003-8c7d-3d0657e3371e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d148ea348514447bf6d5257bc1a61ee'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:40.066253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3bdd88e2-3463-4003-8c7d-3d0657e3371e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d148ea348514447bf6d5257bc1a61ee'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:40.073556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 951/1889] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:39 2022] DELETE /identity/v3/users/0d148ea348514447bf6d5257bc1a61ee => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:40.101347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 939/1890] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:40.106339 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd88e2-3463-4003-8c7d-3d0657e3371e admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:40.107756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3bdd88e2-3463-4003-8c7d-3d0657e3371e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:40.186901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd88e2-3463-4003-8c7d-3d0657e3371e admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:40.191573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9629a-ec12-4099-aa90-4a687de097c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:40.192170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9629a-ec12-4099-aa90-4a687de097c1 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:40.192448 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9629a-ec12-4099-aa90-4a687de097c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:40.192748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9629a-ec12-4099-aa90-4a687de097c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:40.193379 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9629a-ec12-4099-aa90-4a687de097c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:40.195284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9629a-ec12-4099-aa90-4a687de097c1 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:40.247798 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76198117-83c3-408b-bd0a-444071dfe8da None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:40.252597 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-76198117-83c3-408b-bd0a-444071dfe8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:40.273777 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 952/1891] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:40 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:40.278598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9629a-ec12-4099-aa90-4a687de097c1 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:40.324914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76198117-83c3-408b-bd0a-444071dfe8da None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:40.349067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9629a-ec12-4099-aa90-4a687de097c1 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:40.354814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da5a24-4e98-4c7f-ade5-7507fe3255a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:40.356068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da5a24-4e98-4c7f-ade5-7507fe3255a0 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:40.356560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da5a24-4e98-4c7f-ade5-7507fe3255a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:40.356887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da5a24-4e98-4c7f-ade5-7507fe3255a0 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:40.361837 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da5a24-4e98-4c7f-ade5-7507fe3255a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:40.362367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da5a24-4e98-4c7f-ade5-7507fe3255a0 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:40.394869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:40.395510 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:40.396032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:40.396320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] PATH_INFO: `/v3/projects/a385f27e57264b5089585f404c98b0f5` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:40.397974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] RBAC: Authorizing `identity:delete_project(project_id=a385f27e57264b5089585f404c98b0f5)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:40.401540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:40.470816 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 953/1892] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:40 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:52:40.477714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da5a24-4e98-4c7f-ade5-7507fe3255a0 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:40.490902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da5a24-4e98-4c7f-ade5-7507fe3255a0 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:40.507962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a385f27e57264b5089585f404c98b0f5', 'request_id': 'req-953418a7-2e7b-4172-bcc5-fb65a410aeed'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:40.516855 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a385f27e57264b5089585f404c98b0f5', 'request_id': 'req-953418a7-2e7b-4172-bcc5-fb65a410aeed'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:40.565131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a385f27e57264b5089585f404c98b0f5', 'request_id': 'req-953418a7-2e7b-4172-bcc5-fb65a410aeed'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:40.586410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] The token cache is being invalidate because project a385f27e57264b5089585f404c98b0f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:40.589932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:40.589932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:40.590121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 940/1893] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:40 2022] DELETE /identity/v3/projects/a385f27e57264b5089585f404c98b0f5 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:40.593638 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:40.593845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc3bb9-71f5-41be-9b5d-c863263edabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:40.594579 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3bb9-71f5-41be-9b5d-c863263edabe admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:40.594860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3bb9-71f5-41be-9b5d-c863263edabe admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:40.595132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3bb9-71f5-41be-9b5d-c863263edabe admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:40.595337 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:40.595786 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc3bb9-71f5-41be-9b5d-c863263edabe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:40.597562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc3bb9-71f5-41be-9b5d-c863263edabe admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:40.612948 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 954/1894] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:40.614434 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953418a7-2e7b-4172-bcc5-fb65a410aeed admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:40.619271 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec019568-5e00-49d7-8cd9-39ee34bc8968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:40.619999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec019568-5e00-49d7-8cd9-39ee34bc8968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:40.620176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec019568-5e00-49d7-8cd9-39ee34bc8968 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:40.620694 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec019568-5e00-49d7-8cd9-39ee34bc8968 admin admin] PATH_INFO: `/v3/users/411dc9875cfe41fe9d1e6d65665a4687` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:40.622334 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec019568-5e00-49d7-8cd9-39ee34bc8968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=411dc9875cfe41fe9d1e6d65665a4687)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:40.622652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc3bb9-71f5-41be-9b5d-c863263edabe admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:40.624341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec019568-5e00-49d7-8cd9-39ee34bc8968 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:40.624454 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a8bc3bb9-71f5-41be-9b5d-c863263edabe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:40.634095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:40.640280 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc3bb9-71f5-41be-9b5d-c863263edabe admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:40.645081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d873702-30b8-42a9-8b73-380f3b6dc6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:40.649823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d873702-30b8-42a9-8b73-380f3b6dc6ad admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:40.649823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d873702-30b8-42a9-8b73-380f3b6dc6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:40.649823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d873702-30b8-42a9-8b73-380f3b6dc6ad admin admin] PATH_INFO: `/v3/projects/7ad958d9a64a41188fbd4d585ba72818/users/1d80a691569f44028a8781eb775a1e12/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:40.662998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d873702-30b8-42a9-8b73-380f3b6dc6ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad958d9a64a41188fbd4d585ba72818, user_id=1d80a691569f44028a8781eb775a1e12, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:40.665703 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d873702-30b8-42a9-8b73-380f3b6dc6ad admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:40.787593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 955/1895] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:40 2022] PUT /identity/v3/projects/7ad958d9a64a41188fbd4d585ba72818/users/1d80a691569f44028a8781eb775a1e12/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:40.788435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d873702-30b8-42a9-8b73-380f3b6dc6ad admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:40.790951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8d873702-30b8-42a9-8b73-380f3b6dc6ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:40.882043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d873702-30b8-42a9-8b73-380f3b6dc6ad admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:40.889363 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ec019568-5e00-49d7-8cd9-39ee34bc8968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '411dc9875cfe41fe9d1e6d65665a4687'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:40.896846 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11f11f-e933-4302-bf1e-5a52582b0622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:40.897677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11f11f-e933-4302-bf1e-5a52582b0622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:40.897966 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11f11f-e933-4302-bf1e-5a52582b0622 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:40.898243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11f11f-e933-4302-bf1e-5a52582b0622 admin admin] PATH_INFO: `/v3/projects/5e8fab1bef4b41bbbdd2911897b7e904` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:40.899688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11f11f-e933-4302-bf1e-5a52582b0622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e8fab1bef4b41bbbdd2911897b7e904)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:40.901705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11f11f-e933-4302-bf1e-5a52582b0622 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:40.982834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ec019568-5e00-49d7-8cd9-39ee34bc8968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '411dc9875cfe41fe9d1e6d65665a4687'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:40.983596 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ec019568-5e00-49d7-8cd9-39ee34bc8968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '411dc9875cfe41fe9d1e6d65665a4687'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:40.991355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ec019568-5e00-49d7-8cd9-39ee34bc8968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '411dc9875cfe41fe9d1e6d65665a4687'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:41.005070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ec019568-5e00-49d7-8cd9-39ee34bc8968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '411dc9875cfe41fe9d1e6d65665a4687'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:41.016769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 941/1896] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:40 2022] DELETE /identity/v3/users/411dc9875cfe41fe9d1e6d65665a4687 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:41.016769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec019568-5e00-49d7-8cd9-39ee34bc8968 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:41.017168 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ec019568-5e00-49d7-8cd9-39ee34bc8968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:41.032246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8d11f11f-e933-4302-bf1e-5a52582b0622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e8fab1bef4b41bbbdd2911897b7e904', 'request_id': 'req-8d11f11f-e933-4302-bf1e-5a52582b0622'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:41.038980 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8d11f11f-e933-4302-bf1e-5a52582b0622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e8fab1bef4b41bbbdd2911897b7e904', 'request_id': 'req-8d11f11f-e933-4302-bf1e-5a52582b0622'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:41.059255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8d11f11f-e933-4302-bf1e-5a52582b0622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e8fab1bef4b41bbbdd2911897b7e904', 'request_id': 'req-8d11f11f-e933-4302-bf1e-5a52582b0622'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:41.095218 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec019568-5e00-49d7-8cd9-39ee34bc8968 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:41.102329 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6bf76-73d4-43c0-b0b0-92e25edb18c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:41.103025 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6bf76-73d4-43c0-b0b0-92e25edb18c6 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:41.103414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6bf76-73d4-43c0-b0b0-92e25edb18c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:41.103591 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6bf76-73d4-43c0-b0b0-92e25edb18c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:41.105980 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6bf76-73d4-43c0-b0b0-92e25edb18c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:41.109024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6bf76-73d4-43c0-b0b0-92e25edb18c6 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:41.121068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 942/1897] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:41.125252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6bf76-73d4-43c0-b0b0-92e25edb18c6 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:41.131282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8d11f11f-e933-4302-bf1e-5a52582b0622 admin admin] The token cache is being invalidate because project 5e8fab1bef4b41bbbdd2911897b7e904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:41.131588 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8d11f11f-e933-4302-bf1e-5a52582b0622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:41.135142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8d11f11f-e933-4302-bf1e-5a52582b0622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:41.137886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 956/1898] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:40 2022] DELETE /identity/v3/projects/5e8fab1bef4b41bbbdd2911897b7e904 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:41.144900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371c679-a8ef-4c37-b316-4eb34d9a0e84 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:41.145197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371c679-a8ef-4c37-b316-4eb34d9a0e84 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:41.145487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371c679-a8ef-4c37-b316-4eb34d9a0e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:41.158439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6bf76-73d4-43c0-b0b0-92e25edb18c6 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:41.172475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f92c41-3dd9-4eb2-a986-d06bdd6a1e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:41.173364 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92c41-3dd9-4eb2-a986-d06bdd6a1e5d admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:41.173900 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92c41-3dd9-4eb2-a986-d06bdd6a1e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:41.178526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92c41-3dd9-4eb2-a986-d06bdd6a1e5d admin admin] PATH_INFO: `/v3/projects/7ad958d9a64a41188fbd4d585ba72818/users/1d80a691569f44028a8781eb775a1e12/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:41.180241 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f92c41-3dd9-4eb2-a986-d06bdd6a1e5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad958d9a64a41188fbd4d585ba72818, user_id=1d80a691569f44028a8781eb775a1e12, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:41.184627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-8371c679-a8ef-4c37-b316-4eb34d9a0e84 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:41.184884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f92c41-3dd9-4eb2-a986-d06bdd6a1e5d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:41.186018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8371c679-a8ef-4c37-b316-4eb34d9a0e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:41.199487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 943/1899] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:41 2022] PUT /identity/v3/projects/7ad958d9a64a41188fbd4d585ba72818/users/1d80a691569f44028a8781eb775a1e12/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:41.212316 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa476a3-51ff-4726-80bc-064e345578b4 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:41.212966 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa476a3-51ff-4726-80bc-064e345578b4 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:41.213456 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa476a3-51ff-4726-80bc-064e345578b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:41.248473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-2fa476a3-51ff-4726-80bc-064e345578b4 None None] MFA Rules not processed for user `1d80a691569f44028a8781eb775a1e12`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:41.250839 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2fa476a3-51ff-4726-80bc-064e345578b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:41.327583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 957/1900] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:41.335953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04239f-6d59-4259-b991-b6a4569f5661 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:41.336247 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04239f-6d59-4259-b991-b6a4569f5661 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:41.336530 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04239f-6d59-4259-b991-b6a4569f5661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:41.347669 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 944/1901] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:41.360005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa476a3-51ff-4726-80bc-064e345578b4 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:41.361371 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0d04239f-6d59-4259-b991-b6a4569f5661 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:41.361525 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2fa476a3-51ff-4726-80bc-064e345578b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:41.362566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0d04239f-6d59-4259-b991-b6a4569f5661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:41.370435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 958/1902] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:41.374635 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04239f-6d59-4259-b991-b6a4569f5661 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:41.388486 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04239f-6d59-4259-b991-b6a4569f5661 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:41.395694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240e3db-ce8d-4574-a42c-f409c3e378b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:41.396328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240e3db-ce8d-4574-a42c-f409c3e378b3 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:41.396725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240e3db-ce8d-4574-a42c-f409c3e378b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:41.397071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240e3db-ce8d-4574-a42c-f409c3e378b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:41.397817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240e3db-ce8d-4574-a42c-f409c3e378b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:41.398328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240e3db-ce8d-4574-a42c-f409c3e378b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:41.399958 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240e3db-ce8d-4574-a42c-f409c3e378b3 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:41.416498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 959/1903] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:52:41.421950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240e3db-ce8d-4574-a42c-f409c3e378b3 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:41.434316 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240e3db-ce8d-4574-a42c-f409c3e378b3 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:41.441237 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7f689-f809-446e-8c5d-a1dff6974658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:41.442002 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7f689-f809-446e-8c5d-a1dff6974658 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:41.442302 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7f689-f809-446e-8c5d-a1dff6974658 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:41.442577 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7f689-f809-446e-8c5d-a1dff6974658 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:41.443767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7f689-f809-446e-8c5d-a1dff6974658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:41.446097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7f689-f809-446e-8c5d-a1dff6974658 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:41.488039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa476a3-51ff-4726-80bc-064e345578b4 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:41.495187 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c1dbe-2c43-4398-8bcb-142a055baffa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:41.496036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c1dbe-2c43-4398-8bcb-142a055baffa service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:41.496937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c1dbe-2c43-4398-8bcb-142a055baffa service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:41.497226 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c1dbe-2c43-4398-8bcb-142a055baffa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:41.508834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c1dbe-2c43-4398-8bcb-142a055baffa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:41.510518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c1dbe-2c43-4398-8bcb-142a055baffa service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:41.521975 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 945/1904] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:41.582227 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 960/1905] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:41 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:41.583474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:41.586320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c1dbe-2c43-4398-8bcb-142a055baffa service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:41.599872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c1dbe-2c43-4398-8bcb-142a055baffa service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:41.655858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7f689-f809-446e-8c5d-a1dff6974658 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:41.658323 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-81e7f689-f809-446e-8c5d-a1dff6974658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:41.691107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab1924-c56f-41f1-a9ef-2a1cc9167de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:41.691530 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab1924-c56f-41f1-a9ef-2a1cc9167de2 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:41.691991 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab1924-c56f-41f1-a9ef-2a1cc9167de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:41.692435 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab1924-c56f-41f1-a9ef-2a1cc9167de2 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:41.693498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab1924-c56f-41f1-a9ef-2a1cc9167de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:41.695697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab1924-c56f-41f1-a9ef-2a1cc9167de2 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:41.753596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7f689-f809-446e-8c5d-a1dff6974658 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:41.759926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d892a6-f0a5-46d8-b68e-27b45200fe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:41.760953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d892a6-f0a5-46d8-b68e-27b45200fe6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:41.761526 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d892a6-f0a5-46d8-b68e-27b45200fe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:41.762244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d892a6-f0a5-46d8-b68e-27b45200fe6a admin admin] PATH_INFO: `/v3/projects/4454dfb7be6541bfb5346fb4878661f2` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:41.764189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d892a6-f0a5-46d8-b68e-27b45200fe6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4454dfb7be6541bfb5346fb4878661f2)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:41.769009 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d892a6-f0a5-46d8-b68e-27b45200fe6a admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:41.809932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 946/1906] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:41 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:41.811534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab1924-c56f-41f1-a9ef-2a1cc9167de2 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:41.827731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab1924-c56f-41f1-a9ef-2a1cc9167de2 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:41.846728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c602cdf-a64b-46fd-94b8-11b6baf65d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:41.846728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c602cdf-a64b-46fd-94b8-11b6baf65d68 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:41.846728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c602cdf-a64b-46fd-94b8-11b6baf65d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:41.846728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c602cdf-a64b-46fd-94b8-11b6baf65d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:41.847715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c602cdf-a64b-46fd-94b8-11b6baf65d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:41.847715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c602cdf-a64b-46fd-94b8-11b6baf65d68 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:41.865413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 947/1907] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:41.871975 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c602cdf-a64b-46fd-94b8-11b6baf65d68 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:41.897556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c602cdf-a64b-46fd-94b8-11b6baf65d68 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:41.908300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:41.915734 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e360ef-52f9-4c72-9e95-d0a977f22750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:41.916360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e360ef-52f9-4c72-9e95-d0a977f22750 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:41.916639 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e360ef-52f9-4c72-9e95-d0a977f22750 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:41.916963 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e360ef-52f9-4c72-9e95-d0a977f22750 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:41.917698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e360ef-52f9-4c72-9e95-d0a977f22750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:41.920034 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e360ef-52f9-4c72-9e95-d0a977f22750 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:41.931436 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 948/1908] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:41.936446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e360ef-52f9-4c72-9e95-d0a977f22750 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:41.956023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e360ef-52f9-4c72-9e95-d0a977f22750 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:41.966105 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-60d892a6-f0a5-46d8-b68e-27b45200fe6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4454dfb7be6541bfb5346fb4878661f2', 'request_id': 'req-60d892a6-f0a5-46d8-b68e-27b45200fe6a'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:41.981548 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-60d892a6-f0a5-46d8-b68e-27b45200fe6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4454dfb7be6541bfb5346fb4878661f2', 'request_id': 'req-60d892a6-f0a5-46d8-b68e-27b45200fe6a'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:41.993997 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-60d892a6-f0a5-46d8-b68e-27b45200fe6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4454dfb7be6541bfb5346fb4878661f2', 'request_id': 'req-60d892a6-f0a5-46d8-b68e-27b45200fe6a'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:42.056209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a071eb-408a-4880-aa79-f60119defdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:42.057064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a071eb-408a-4880-aa79-f60119defdbc admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:42.057180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-60d892a6-f0a5-46d8-b68e-27b45200fe6a admin admin] The token cache is being invalidate because project 4454dfb7be6541bfb5346fb4878661f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:42.057617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-60d892a6-f0a5-46d8-b68e-27b45200fe6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:42.057704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a071eb-408a-4880-aa79-f60119defdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:42.057877 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a071eb-408a-4880-aa79-f60119defdbc admin admin] PATH_INFO: `/v3/projects/4decf63db0594949b321e3450a122100/users/83fa9e38ff6d4620aae391266c82fa1c/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:42.059672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-60d892a6-f0a5-46d8-b68e-27b45200fe6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:42.068426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 961/1909] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:41 2022] DELETE /identity/v3/projects/4454dfb7be6541bfb5346fb4878661f2 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:42.071897 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e54a30-26dd-4d0c-9fd4-fb5db9e84aa0 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:42.072244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e54a30-26dd-4d0c-9fd4-fb5db9e84aa0 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:42.072535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e54a30-26dd-4d0c-9fd4-fb5db9e84aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:42.092345 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a071eb-408a-4880-aa79-f60119defdbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4decf63db0594949b321e3450a122100, user_id=83fa9e38ff6d4620aae391266c82fa1c, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:42.100375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a071eb-408a-4880-aa79-f60119defdbc admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:42.115233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a8e54a30-26dd-4d0c-9fd4-fb5db9e84aa0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:42.117069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a8e54a30-26dd-4d0c-9fd4-fb5db9e84aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:42.127612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 962/1910] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:42.135865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b68ca-396d-4b37-8ae0-591fc578264d None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:42.136161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b68ca-396d-4b37-8ae0-591fc578264d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:42.136444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b68ca-396d-4b37-8ae0-591fc578264d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:42.159299 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-288b68ca-396d-4b37-8ae0-591fc578264d None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:42.164486 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-288b68ca-396d-4b37-8ae0-591fc578264d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:42.171232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 963/1911] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:42.175315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b68ca-396d-4b37-8ae0-591fc578264d None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:42.184667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 949/1912] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:41 2022] PUT /identity/v3/projects/4decf63db0594949b321e3450a122100/users/83fa9e38ff6d4620aae391266c82fa1c/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:42.188366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b68ca-396d-4b37-8ae0-591fc578264d None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:42.192443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8428-e37c-4a1f-9f1a-5eeaed2eb0d3 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:42.192870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8428-e37c-4a1f-9f1a-5eeaed2eb0d3 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:42.193250 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8428-e37c-4a1f-9f1a-5eeaed2eb0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:42.241130 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-53ee8428-e37c-4a1f-9f1a-5eeaed2eb0d3 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:42.242247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-53ee8428-e37c-4a1f-9f1a-5eeaed2eb0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:42.275050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8ebe5-9a82-4562-bc39-01fbf4ba542c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:42.276161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8ebe5-9a82-4562-bc39-01fbf4ba542c admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:42.276837 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8ebe5-9a82-4562-bc39-01fbf4ba542c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:42.277571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8ebe5-9a82-4562-bc39-01fbf4ba542c admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:42.279147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8ebe5-9a82-4562-bc39-01fbf4ba542c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:42.280408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8ebe5-9a82-4562-bc39-01fbf4ba542c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:42.282615 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8ebe5-9a82-4562-bc39-01fbf4ba542c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:42.309323 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 964/1913] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:52:42.310333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8ebe5-9a82-4562-bc39-01fbf4ba542c admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:42.324119 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8ebe5-9a82-4562-bc39-01fbf4ba542c admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:42.341407 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fd039-63c8-4ef6-affc-94f73835691b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:42.342379 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fd039-63c8-4ef6-affc-94f73835691b admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:42.343081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fd039-63c8-4ef6-affc-94f73835691b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:42.343441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fd039-63c8-4ef6-affc-94f73835691b admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:42.344184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fd039-63c8-4ef6-affc-94f73835691b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:42.346286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fd039-63c8-4ef6-affc-94f73835691b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:42.349701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 950/1914] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:42.351059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee8428-e37c-4a1f-9f1a-5eeaed2eb0d3 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:42.352514 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-53ee8428-e37c-4a1f-9f1a-5eeaed2eb0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:42.364993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee8428-e37c-4a1f-9f1a-5eeaed2eb0d3 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:42.370543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc15f54-139f-4299-9596-a1f0ed5b0779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:42.374519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc15f54-139f-4299-9596-a1f0ed5b0779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:42.374519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc15f54-139f-4299-9596-a1f0ed5b0779 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:42.374519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc15f54-139f-4299-9596-a1f0ed5b0779 admin admin] PATH_INFO: `/v3/users/3ddd62d59df24013b92fa85a51718d57` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:42.374519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc15f54-139f-4299-9596-a1f0ed5b0779 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ddd62d59df24013b92fa85a51718d57)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:42.376091 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc15f54-139f-4299-9596-a1f0ed5b0779 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:42.463549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 965/1915] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:42 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:42.464393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fd039-63c8-4ef6-affc-94f73835691b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:42.466739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c70fd039-63c8-4ef6-affc-94f73835691b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:42.529277 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fd039-63c8-4ef6-affc-94f73835691b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:42.533250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc9a8d-7b78-426c-8771-bcda57877e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:42.534929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc9a8d-7b78-426c-8771-bcda57877e81 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:42.534929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc9a8d-7b78-426c-8771-bcda57877e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:42.534929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc9a8d-7b78-426c-8771-bcda57877e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:42.601669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc9a8d-7b78-426c-8771-bcda57877e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:42.603357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc9a8d-7b78-426c-8771-bcda57877e81 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:42.616241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 966/1916] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:42.617400 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc9a8d-7b78-426c-8771-bcda57877e81 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:42.633210 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc9a8d-7b78-426c-8771-bcda57877e81 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:42.639319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199a229-98d4-407b-a3c8-423aa5dcfcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:42.640206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199a229-98d4-407b-a3c8-423aa5dcfcdb admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:42.640812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199a229-98d4-407b-a3c8-423aa5dcfcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:42.641308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199a229-98d4-407b-a3c8-423aa5dcfcdb admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:42.642287 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199a229-98d4-407b-a3c8-423aa5dcfcdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:42.645335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199a229-98d4-407b-a3c8-423aa5dcfcdb admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:42.651260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-afc15f54-139f-4299-9596-a1f0ed5b0779 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ddd62d59df24013b92fa85a51718d57'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:42.787996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-afc15f54-139f-4299-9596-a1f0ed5b0779 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ddd62d59df24013b92fa85a51718d57'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:42.788598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-afc15f54-139f-4299-9596-a1f0ed5b0779 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ddd62d59df24013b92fa85a51718d57'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:42.797065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-afc15f54-139f-4299-9596-a1f0ed5b0779 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ddd62d59df24013b92fa85a51718d57'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:42.805597 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 967/1917] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:42 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:42.810731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199a229-98d4-407b-a3c8-423aa5dcfcdb admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:42.817407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:42.818700 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-afc15f54-139f-4299-9596-a1f0ed5b0779 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ddd62d59df24013b92fa85a51718d57'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:42.831353 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 951/1918] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:42 2022] DELETE /identity/v3/users/3ddd62d59df24013b92fa85a51718d57 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:42.833909 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8a324-3df9-4ab0-bc18-c07caf1b14ba None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:42.834405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8a324-3df9-4ab0-bc18-c07caf1b14ba None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:42.835314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8a324-3df9-4ab0-bc18-c07caf1b14ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:42.843259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199a229-98d4-407b-a3c8-423aa5dcfcdb admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:42.861547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-22d8a324-3df9-4ab0-bc18-c07caf1b14ba None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:42.863013 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-22d8a324-3df9-4ab0-bc18-c07caf1b14ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:42.924005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:42.967146 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1791812-13c1-4128-859d-fe35431570fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:42.968070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1791812-13c1-4128-859d-fe35431570fe admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:42.968487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1791812-13c1-4128-859d-fe35431570fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:42.968869 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1791812-13c1-4128-859d-fe35431570fe admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:42.970081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1791812-13c1-4128-859d-fe35431570fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:42.971401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1791812-13c1-4128-859d-fe35431570fe admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:42.977006 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 952/1919] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:42.986526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8a324-3df9-4ab0-bc18-c07caf1b14ba None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:42.988409 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-22d8a324-3df9-4ab0-bc18-c07caf1b14ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:42.992878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 968/1920] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:42.995287 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1791812-13c1-4128-859d-fe35431570fe admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:43.000005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b1791812-13c1-4128-859d-fe35431570fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:43.024771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1791812-13c1-4128-859d-fe35431570fe admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:43.030188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647dc8c-6d44-488e-8446-eda0986ac8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:43.030810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647dc8c-6d44-488e-8446-eda0986ac8bb admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:43.031095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647dc8c-6d44-488e-8446-eda0986ac8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:43.031374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647dc8c-6d44-488e-8446-eda0986ac8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:43.032028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647dc8c-6d44-488e-8446-eda0986ac8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:43.033790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647dc8c-6d44-488e-8446-eda0986ac8bb admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:43.050705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 969/1921] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:43.051941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647dc8c-6d44-488e-8446-eda0986ac8bb admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:43.062476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647dc8c-6d44-488e-8446-eda0986ac8bb admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:43.069705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc587a33-f674-407c-a840-63111362191d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:43.069705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc587a33-f674-407c-a840-63111362191d admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:43.069705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc587a33-f674-407c-a840-63111362191d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:43.069705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc587a33-f674-407c-a840-63111362191d admin admin] PATH_INFO: `/v3/projects/ce2ccfbe0e5141d0be1c854bb7be312d/users/8b1479e196844d6cba238f34924068d4/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:43.086804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc587a33-f674-407c-a840-63111362191d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2ccfbe0e5141d0be1c854bb7be312d, user_id=8b1479e196844d6cba238f34924068d4, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:43.089409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc587a33-f674-407c-a840-63111362191d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:43.091848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8a324-3df9-4ab0-bc18-c07caf1b14ba None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:43.097005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f457e4b-2899-4758-a8df-820233f0568e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:43.097731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f457e4b-2899-4758-a8df-820233f0568e service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:43.098020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f457e4b-2899-4758-a8df-820233f0568e service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:43.098297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f457e4b-2899-4758-a8df-820233f0568e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:43.109701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f457e4b-2899-4758-a8df-820233f0568e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:43.111202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f457e4b-2899-4758-a8df-820233f0568e service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:43.122904 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 953/1922] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:43.182767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 970/1923] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:43 2022] PUT /identity/v3/projects/ce2ccfbe0e5141d0be1c854bb7be312d/users/8b1479e196844d6cba238f34924068d4/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:43.194234 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a716488-1108-4ce3-8da2-383f67b4cd0d None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:43.194608 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a716488-1108-4ce3-8da2-383f67b4cd0d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:43.195025 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a716488-1108-4ce3-8da2-383f67b4cd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:43.226526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-1a716488-1108-4ce3-8da2-383f67b4cd0d None None] MFA Rules not processed for user `8b1479e196844d6cba238f34924068d4`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:43.227853 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-1a716488-1108-4ce3-8da2-383f67b4cd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:43.317336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 954/1924] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:43.328147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc587a33-f674-407c-a840-63111362191d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:43.328147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-dc587a33-f674-407c-a840-63111362191d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:43.416046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc587a33-f674-407c-a840-63111362191d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:43.424076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab9aeb-339f-4674-9599-a9c6ae7cd079 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38de805ba5eb4aa58cf95d2082c2d216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:43.425169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab9aeb-339f-4674-9599-a9c6ae7cd079 service glance] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:43.425627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab9aeb-339f-4674-9599-a9c6ae7cd079 service glance] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:43.426229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab9aeb-339f-4674-9599-a9c6ae7cd079 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:43.440008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab9aeb-339f-4674-9599-a9c6ae7cd079 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:43.441803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab9aeb-339f-4674-9599-a9c6ae7cd079 service glance] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:43.454018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 971/1925] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:43.822867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:43.961128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a716488-1108-4ce3-8da2-383f67b4cd0d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:43.962917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-1a716488-1108-4ce3-8da2-383f67b4cd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:44.023973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:44.057183 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a716488-1108-4ce3-8da2-383f67b4cd0d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:44.062305 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55e172-4993-427f-853f-dc0901409c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:44.062936 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e172-4993-427f-853f-dc0901409c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:44.063214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e172-4993-427f-853f-dc0901409c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:44.063486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e172-4993-427f-853f-dc0901409c3c admin admin] PATH_INFO: `/v3/projects/5c9a7f07d478479fa050eee69bc79516` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:44.064595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55e172-4993-427f-853f-dc0901409c3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c9a7f07d478479fa050eee69bc79516)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:44.066566 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55e172-4993-427f-853f-dc0901409c3c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:44.204243 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-af55e172-4993-427f-853f-dc0901409c3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c9a7f07d478479fa050eee69bc79516', 'request_id': 'req-af55e172-4993-427f-853f-dc0901409c3c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:44.211227 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-af55e172-4993-427f-853f-dc0901409c3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c9a7f07d478479fa050eee69bc79516', 'request_id': 'req-af55e172-4993-427f-853f-dc0901409c3c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:44.290474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-af55e172-4993-427f-853f-dc0901409c3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c9a7f07d478479fa050eee69bc79516', 'request_id': 'req-af55e172-4993-427f-853f-dc0901409c3c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:44.304867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-af55e172-4993-427f-853f-dc0901409c3c admin admin] The token cache is being invalidate because project 5c9a7f07d478479fa050eee69bc79516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:44.305186 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-af55e172-4993-427f-853f-dc0901409c3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:44.305925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-af55e172-4993-427f-853f-dc0901409c3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:44.307698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 955/1926] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:43 2022] DELETE /identity/v3/projects/5c9a7f07d478479fa050eee69bc79516 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:44.311629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab9aeb-339f-4674-9599-a9c6ae7cd079 service glance] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:44.323368 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab9aeb-339f-4674-9599-a9c6ae7cd079 service glance] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:44.399882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ebec2-a246-40df-b2ea-4395ab8852a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:44.400526 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ebec2-a246-40df-b2ea-4395ab8852a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:44.400857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ebec2-a246-40df-b2ea-4395ab8852a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:44.401172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ebec2-a246-40df-b2ea-4395ab8852a3 admin admin] PATH_INFO: `/v3/users/1d80a691569f44028a8781eb775a1e12` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:44.404763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ebec2-a246-40df-b2ea-4395ab8852a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d80a691569f44028a8781eb775a1e12)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:44.406165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ebec2-a246-40df-b2ea-4395ab8852a3 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:44.723869 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-594ebec2-a246-40df-b2ea-4395ab8852a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d80a691569f44028a8781eb775a1e12'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:44.724403 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7526e-8a5b-4346-8e08-6e1714ec9a93 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:44.725723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7526e-8a5b-4346-8e08-6e1714ec9a93 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:44.726692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7526e-8a5b-4346-8e08-6e1714ec9a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:44.739835 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-594ebec2-a246-40df-b2ea-4395ab8852a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d80a691569f44028a8781eb775a1e12'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:44.802511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-c2d7526e-8a5b-4346-8e08-6e1714ec9a93 None None] MFA Rules not processed for user `83fa9e38ff6d4620aae391266c82fa1c`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:44.804063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c2d7526e-8a5b-4346-8e08-6e1714ec9a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:44.879141 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-594ebec2-a246-40df-b2ea-4395ab8852a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d80a691569f44028a8781eb775a1e12'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:44.883674 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-594ebec2-a246-40df-b2ea-4395ab8852a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d80a691569f44028a8781eb775a1e12'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:44.914024 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-594ebec2-a246-40df-b2ea-4395ab8852a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d80a691569f44028a8781eb775a1e12'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:44.924328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 972/1927] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:44 2022] DELETE /identity/v3/users/1d80a691569f44028a8781eb775a1e12 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:44.938475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 956/1928] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:44.956869 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ebec2-a246-40df-b2ea-4395ab8852a3 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:44.958540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-594ebec2-a246-40df-b2ea-4395ab8852a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:45.052227 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ebec2-a246-40df-b2ea-4395ab8852a3 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:45.057063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e267def-cdcf-4daa-8918-036025e2e0c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:45.058809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e267def-cdcf-4daa-8918-036025e2e0c5 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:45.059196 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e267def-cdcf-4daa-8918-036025e2e0c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:45.059700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e267def-cdcf-4daa-8918-036025e2e0c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:45.134713 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e267def-cdcf-4daa-8918-036025e2e0c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:45.136296 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e267def-cdcf-4daa-8918-036025e2e0c5 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:45.149894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 973/1929] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:45.163059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:45.275056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:45.723362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7526e-8a5b-4346-8e08-6e1714ec9a93 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:45.725121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c2d7526e-8a5b-4346-8e08-6e1714ec9a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:45.754217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7526e-8a5b-4346-8e08-6e1714ec9a93 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:45.828187 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fd931-caf4-4f59-aeee-06ae29ec203b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:45.828867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fd931-caf4-4f59-aeee-06ae29ec203b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:45.829156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fd931-caf4-4f59-aeee-06ae29ec203b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:45.829690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fd931-caf4-4f59-aeee-06ae29ec203b admin admin] PATH_INFO: `/v3/projects/7ad958d9a64a41188fbd4d585ba72818` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:45.830878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fd931-caf4-4f59-aeee-06ae29ec203b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ad958d9a64a41188fbd4d585ba72818)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:45.832847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fd931-caf4-4f59-aeee-06ae29ec203b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:45.999169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-da9fd931-caf4-4f59-aeee-06ae29ec203b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ad958d9a64a41188fbd4d585ba72818', 'request_id': 'req-da9fd931-caf4-4f59-aeee-06ae29ec203b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:46.008147 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-da9fd931-caf4-4f59-aeee-06ae29ec203b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ad958d9a64a41188fbd4d585ba72818', 'request_id': 'req-da9fd931-caf4-4f59-aeee-06ae29ec203b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:46.142243 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-da9fd931-caf4-4f59-aeee-06ae29ec203b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ad958d9a64a41188fbd4d585ba72818', 'request_id': 'req-da9fd931-caf4-4f59-aeee-06ae29ec203b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:46.154220 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-da9fd931-caf4-4f59-aeee-06ae29ec203b admin admin] The token cache is being invalidate because project 7ad958d9a64a41188fbd4d585ba72818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:46.154510 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-da9fd931-caf4-4f59-aeee-06ae29ec203b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:46.155144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-da9fd931-caf4-4f59-aeee-06ae29ec203b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:46.156663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 957/1930] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:45 2022] DELETE /identity/v3/projects/7ad958d9a64a41188fbd4d585ba72818 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:46.161675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699bf56-59cb-4609-a6b8-f19b3057e154 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:46.162192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699bf56-59cb-4609-a6b8-f19b3057e154 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:46.162290 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699bf56-59cb-4609-a6b8-f19b3057e154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:46.181888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0699bf56-59cb-4609-a6b8-f19b3057e154 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:46.183134 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0699bf56-59cb-4609-a6b8-f19b3057e154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:46.263395 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 974/1931] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:46.276622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433b9a7-e3f8-448c-be5f-f214525611aa None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:46.276622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433b9a7-e3f8-448c-be5f-f214525611aa None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:46.276622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433b9a7-e3f8-448c-be5f-f214525611aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:46.292259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-4433b9a7-e3f8-448c-be5f-f214525611aa None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:46.293691 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4433b9a7-e3f8-448c-be5f-f214525611aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:46.301869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 958/1932] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:46.309662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699bf56-59cb-4609-a6b8-f19b3057e154 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:46.319079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699bf56-59cb-4609-a6b8-f19b3057e154 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:46.323659 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9fe1b-01b7-4c42-924c-ded6fe0c62c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:46.324856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9fe1b-01b7-4c42-924c-ded6fe0c62c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:46.325143 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9fe1b-01b7-4c42-924c-ded6fe0c62c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:46.325416 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9fe1b-01b7-4c42-924c-ded6fe0c62c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:46.326062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a9fe1b-01b7-4c42-924c-ded6fe0c62c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:46.326369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a9fe1b-01b7-4c42-924c-ded6fe0c62c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:46.327963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a9fe1b-01b7-4c42-924c-ded6fe0c62c5 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:46.340259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 975/1933] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:52:46.346465 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433b9a7-e3f8-448c-be5f-f214525611aa None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:46.356081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433b9a7-e3f8-448c-be5f-f214525611aa None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:46.368157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189ae97-1f59-405d-b6ca-8df2e9139e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:46.369418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ae97-1f59-405d-b6ca-8df2e9139e5b admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:46.369825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ae97-1f59-405d-b6ca-8df2e9139e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:46.370421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ae97-1f59-405d-b6ca-8df2e9139e5b admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:46.371972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189ae97-1f59-405d-b6ca-8df2e9139e5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:46.376147 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189ae97-1f59-405d-b6ca-8df2e9139e5b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:46.389633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:46.492730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 959/1934] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:46 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:46.497411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9fe1b-01b7-4c42-924c-ded6fe0c62c5 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:46.513723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9fe1b-01b7-4c42-924c-ded6fe0c62c5 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:46.583249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:46.596226 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073bff3-117a-4e6c-9761-6513beea9dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:46.597052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073bff3-117a-4e6c-9761-6513beea9dcc admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:46.597439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073bff3-117a-4e6c-9761-6513beea9dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:46.597939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073bff3-117a-4e6c-9761-6513beea9dcc admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:46.601112 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073bff3-117a-4e6c-9761-6513beea9dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:46.603283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073bff3-117a-4e6c-9761-6513beea9dcc admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:46.711453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 976/1935] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:46 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:46.714633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189ae97-1f59-405d-b6ca-8df2e9139e5b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:46.724883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189ae97-1f59-405d-b6ca-8df2e9139e5b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:46.731522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5630e-ac39-4ce6-a05d-b53aef2f08e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:46.732128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5630e-ac39-4ce6-a05d-b53aef2f08e3 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:46.732407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5630e-ac39-4ce6-a05d-b53aef2f08e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:46.732710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5630e-ac39-4ce6-a05d-b53aef2f08e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:46.733361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5630e-ac39-4ce6-a05d-b53aef2f08e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:46.734941 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5630e-ac39-4ce6-a05d-b53aef2f08e3 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:46.745061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 960/1936] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:46.749981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073bff3-117a-4e6c-9761-6513beea9dcc admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:46.760064 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073bff3-117a-4e6c-9761-6513beea9dcc admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:46.765134 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884adccd-bd43-41ce-9753-c126b5c2117d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:46.765726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884adccd-bd43-41ce-9753-c126b5c2117d admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:46.766007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884adccd-bd43-41ce-9753-c126b5c2117d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:46.766277 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884adccd-bd43-41ce-9753-c126b5c2117d admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:46.766908 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884adccd-bd43-41ce-9753-c126b5c2117d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:46.768493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884adccd-bd43-41ce-9753-c126b5c2117d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:46.779352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 977/1937] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:46.783496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5630e-ac39-4ce6-a05d-b53aef2f08e3 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:46.795099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5630e-ac39-4ce6-a05d-b53aef2f08e3 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:46.800071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff945d39-0059-488f-9a5f-4d5b51f02520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:46.800882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff945d39-0059-488f-9a5f-4d5b51f02520 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:46.801167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff945d39-0059-488f-9a5f-4d5b51f02520 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:46.805961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff945d39-0059-488f-9a5f-4d5b51f02520 admin admin] PATH_INFO: `/v3/projects/fbc1edc7685f4b5e8b896244bb6b9479/users/0442a0ca5ea24213bbd0e1e1ee2f5583/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:46.816655 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff945d39-0059-488f-9a5f-4d5b51f02520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc1edc7685f4b5e8b896244bb6b9479, user_id=0442a0ca5ea24213bbd0e1e1ee2f5583, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:46.819373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff945d39-0059-488f-9a5f-4d5b51f02520 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:46.955069 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 961/1938] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:46 2022] PUT /identity/v3/projects/fbc1edc7685f4b5e8b896244bb6b9479/users/0442a0ca5ea24213bbd0e1e1ee2f5583/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:46.966697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627c5c2-c7dc-404c-bd92-aca17bb4ac04 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:46.966986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627c5c2-c7dc-404c-bd92-aca17bb4ac04 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:46.967265 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627c5c2-c7dc-404c-bd92-aca17bb4ac04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:46.996462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-3627c5c2-c7dc-404c-bd92-aca17bb4ac04 None None] MFA Rules not processed for user `0442a0ca5ea24213bbd0e1e1ee2f5583`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:46.997682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3627c5c2-c7dc-404c-bd92-aca17bb4ac04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:47.066139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 978/1939] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:47.070401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff945d39-0059-488f-9a5f-4d5b51f02520 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:47.193458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff945d39-0059-488f-9a5f-4d5b51f02520 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:47.199777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6621cc-5d17-4a8a-9e70-93ceecd08904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:47.200792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6621cc-5d17-4a8a-9e70-93ceecd08904 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:47.201194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6621cc-5d17-4a8a-9e70-93ceecd08904 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:47.201715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6621cc-5d17-4a8a-9e70-93ceecd08904 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:47.202609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6621cc-5d17-4a8a-9e70-93ceecd08904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:47.205155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6621cc-5d17-4a8a-9e70-93ceecd08904 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:47.251425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:47.305300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:47.319011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 962/1940] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:47 2022] POST /identity/v3/projects => generated 383 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:47.323375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627c5c2-c7dc-404c-bd92-aca17bb4ac04 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:47.344561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627c5c2-c7dc-404c-bd92-aca17bb4ac04 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:47.370501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70c0d9-d0f3-4151-9e78-8fe1bac86b24 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:47.371712 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70c0d9-d0f3-4151-9e78-8fe1bac86b24 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:47.372222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70c0d9-d0f3-4151-9e78-8fe1bac86b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:47.396246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-6f70c0d9-d0f3-4151-9e78-8fe1bac86b24 None None] MFA Rules not processed for user `83fa9e38ff6d4620aae391266c82fa1c`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:47.397898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6f70c0d9-d0f3-4151-9e78-8fe1bac86b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:47.464304 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aed93e-218d-46c5-ab6e-4231d7d71256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:47.465565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aed93e-218d-46c5-ab6e-4231d7d71256 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:47.466507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aed93e-218d-46c5-ab6e-4231d7d71256 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:47.466951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aed93e-218d-46c5-ab6e-4231d7d71256 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:47.467546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aed93e-218d-46c5-ab6e-4231d7d71256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:47.469504 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aed93e-218d-46c5-ab6e-4231d7d71256 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:47.504255 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 963/1941] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:47.545072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:47.552963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 979/1942] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:47 2022] POST /identity/v3/users => generated 298 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:47.558143 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70c0d9-d0f3-4151-9e78-8fe1bac86b24 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:47.573375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70c0d9-d0f3-4151-9e78-8fe1bac86b24 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:47.581071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f85d57-fdc6-4dd7-a031-ca758e1bc540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:47.581696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f85d57-fdc6-4dd7-a031-ca758e1bc540 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:47.581982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f85d57-fdc6-4dd7-a031-ca758e1bc540 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:47.582259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f85d57-fdc6-4dd7-a031-ca758e1bc540 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:47.583939 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f85d57-fdc6-4dd7-a031-ca758e1bc540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:47.592293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f85d57-fdc6-4dd7-a031-ca758e1bc540 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:47.617020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 964/1943] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:52:47.617795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aed93e-218d-46c5-ab6e-4231d7d71256 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:47.630471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aed93e-218d-46c5-ab6e-4231d7d71256 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:47.635951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b8914-873b-4d25-8106-d6843e06f67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:47.636752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b8914-873b-4d25-8106-d6843e06f67f admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:47.637045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b8914-873b-4d25-8106-d6843e06f67f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:47.637411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b8914-873b-4d25-8106-d6843e06f67f admin admin] PATH_INFO: `/v3/projects/1c0cfb77dd07465cb76d9619124d6d4a/users/27ce124d578b47c1a723868899c1933b/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:47.655969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b8914-873b-4d25-8106-d6843e06f67f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c0cfb77dd07465cb76d9619124d6d4a, user_id=27ce124d578b47c1a723868899c1933b, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:47.658950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b8914-873b-4d25-8106-d6843e06f67f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:47.728491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 980/1944] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:47 2022] PUT /identity/v3/projects/1c0cfb77dd07465cb76d9619124d6d4a/users/27ce124d578b47c1a723868899c1933b/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:47.733555 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f85d57-fdc6-4dd7-a031-ca758e1bc540 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:47.746647 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f85d57-fdc6-4dd7-a031-ca758e1bc540 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:47.784893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:47.825819 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101dd13-d953-42da-b0ec-bcfc6bbfac51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:47.826655 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101dd13-d953-42da-b0ec-bcfc6bbfac51 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:47.826944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101dd13-d953-42da-b0ec-bcfc6bbfac51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:47.827908 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101dd13-d953-42da-b0ec-bcfc6bbfac51 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:47.828571 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101dd13-d953-42da-b0ec-bcfc6bbfac51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:47.840197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101dd13-d953-42da-b0ec-bcfc6bbfac51 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:47.842066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 965/1945] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:47.846101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b8914-873b-4d25-8106-d6843e06f67f admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:47.861529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b8914-873b-4d25-8106-d6843e06f67f admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:47.871639 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88346930-ce26-4763-bc97-a6c2af6483eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:47.872608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88346930-ce26-4763-bc97-a6c2af6483eb admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:47.872921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88346930-ce26-4763-bc97-a6c2af6483eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:47.873371 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88346930-ce26-4763-bc97-a6c2af6483eb admin admin] PATH_INFO: `/v3/projects/1c0cfb77dd07465cb76d9619124d6d4a/users/27ce124d578b47c1a723868899c1933b/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:47.875492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88346930-ce26-4763-bc97-a6c2af6483eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c0cfb77dd07465cb76d9619124d6d4a, user_id=27ce124d578b47c1a723868899c1933b, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:47.882570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88346930-ce26-4763-bc97-a6c2af6483eb admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:47.912730 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 981/1946] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:47 2022] PUT /identity/v3/projects/1c0cfb77dd07465cb76d9619124d6d4a/users/27ce124d578b47c1a723868899c1933b/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:47.917161 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e2fbb-a394-47e5-b61f-1a1e69d5fc59 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:47.918024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e2fbb-a394-47e5-b61f-1a1e69d5fc59 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:47.918670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e2fbb-a394-47e5-b61f-1a1e69d5fc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:47.953238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-816e2fbb-a394-47e5-b61f-1a1e69d5fc59 None None] MFA Rules not processed for user `27ce124d578b47c1a723868899c1933b`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:47.954711 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-816e2fbb-a394-47e5-b61f-1a1e69d5fc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:48.038503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 966/1947] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:48.074593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88346930-ce26-4763-bc97-a6c2af6483eb admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:48.076426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-88346930-ce26-4763-bc97-a6c2af6483eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:48.157975 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:48.161369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88346930-ce26-4763-bc97-a6c2af6483eb admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:48.166049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a8194-8321-47b6-89ec-c0a8c915fd0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:48.166459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a8194-8321-47b6-89ec-c0a8c915fd0b service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:48.166799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a8194-8321-47b6-89ec-c0a8c915fd0b service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:48.168308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a8194-8321-47b6-89ec-c0a8c915fd0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:48.179475 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a8194-8321-47b6-89ec-c0a8c915fd0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:48.181660 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a8194-8321-47b6-89ec-c0a8c915fd0b service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:48.194049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 982/1948] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:48.246317 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:48.495115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e2fbb-a394-47e5-b61f-1a1e69d5fc59 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:48.497426 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-816e2fbb-a394-47e5-b61f-1a1e69d5fc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:48.516216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e2fbb-a394-47e5-b61f-1a1e69d5fc59 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:48.522431 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc37ed-ea3e-419e-a862-d1e553fa0362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:48.523613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc37ed-ea3e-419e-a862-d1e553fa0362 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:48.524224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc37ed-ea3e-419e-a862-d1e553fa0362 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:48.524840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc37ed-ea3e-419e-a862-d1e553fa0362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:48.527761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:48.561743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:48.614296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc37ed-ea3e-419e-a862-d1e553fa0362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:48.615790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc37ed-ea3e-419e-a862-d1e553fa0362 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:48.628873 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 967/1949] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:48.864180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:49.178600 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a8194-8321-47b6-89ec-c0a8c915fd0b service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:49.179474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc37ed-ea3e-419e-a862-d1e553fa0362 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:49.180208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-587a8194-8321-47b6-89ec-c0a8c915fd0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:49.184680 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:49.264121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc37ed-ea3e-419e-a862-d1e553fa0362 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:49.267608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a8194-8321-47b6-89ec-c0a8c915fd0b service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:49.268902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca50a2-fe4d-4ff1-a2aa-cc9ea76ca16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:49.271142 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca50a2-fe4d-4ff1-a2aa-cc9ea76ca16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:49.271142 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca50a2-fe4d-4ff1-a2aa-cc9ea76ca16e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:49.271142 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca50a2-fe4d-4ff1-a2aa-cc9ea76ca16e admin admin] PATH_INFO: `/v3/users/0442a0ca5ea24213bbd0e1e1ee2f5583` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:49.271472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca50a2-fe4d-4ff1-a2aa-cc9ea76ca16e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0442a0ca5ea24213bbd0e1e1ee2f5583)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:49.272867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6586a16-d5d8-407e-9f5a-791c59e3c263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:49.273372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca50a2-fe4d-4ff1-a2aa-cc9ea76ca16e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:49.273493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6586a16-d5d8-407e-9f5a-791c59e3c263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:49.273776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6586a16-d5d8-407e-9f5a-791c59e3c263 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:49.274051 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6586a16-d5d8-407e-9f5a-791c59e3c263 admin admin] PATH_INFO: `/v3/users/fd15e9fec2af4261990e3b6bcb5dea8c` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:49.275183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6586a16-d5d8-407e-9f5a-791c59e3c263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd15e9fec2af4261990e3b6bcb5dea8c)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:49.277877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6586a16-d5d8-407e-9f5a-791c59e3c263 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:49.505798 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a3ca50a2-fe4d-4ff1-a2aa-cc9ea76ca16e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0442a0ca5ea24213bbd0e1e1ee2f5583'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:49.554434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f6586a16-d5d8-407e-9f5a-791c59e3c263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd15e9fec2af4261990e3b6bcb5dea8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:49.560018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:49.567736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f6586a16-d5d8-407e-9f5a-791c59e3c263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd15e9fec2af4261990e3b6bcb5dea8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:49.587918 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a3ca50a2-fe4d-4ff1-a2aa-cc9ea76ca16e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0442a0ca5ea24213bbd0e1e1ee2f5583'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:49.588561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a3ca50a2-fe4d-4ff1-a2aa-cc9ea76ca16e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0442a0ca5ea24213bbd0e1e1ee2f5583'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:49.600680 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a3ca50a2-fe4d-4ff1-a2aa-cc9ea76ca16e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0442a0ca5ea24213bbd0e1e1ee2f5583'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:49.609209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a3ca50a2-fe4d-4ff1-a2aa-cc9ea76ca16e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0442a0ca5ea24213bbd0e1e1ee2f5583'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:49.615121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 968/1950] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:49 2022] DELETE /identity/v3/users/0442a0ca5ea24213bbd0e1e1ee2f5583 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:49.619395 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5251ae-44ac-40d6-a456-b911b2a998cb None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:49.619687 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5251ae-44ac-40d6-a456-b911b2a998cb None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:49.619969 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5251ae-44ac-40d6-a456-b911b2a998cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:49.639661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-7b5251ae-44ac-40d6-a456-b911b2a998cb None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:49.640924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-7b5251ae-44ac-40d6-a456-b911b2a998cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:49.673651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f6586a16-d5d8-407e-9f5a-791c59e3c263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd15e9fec2af4261990e3b6bcb5dea8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:49.674615 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f6586a16-d5d8-407e-9f5a-791c59e3c263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd15e9fec2af4261990e3b6bcb5dea8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:49.695647 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f6586a16-d5d8-407e-9f5a-791c59e3c263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd15e9fec2af4261990e3b6bcb5dea8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:49.704698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 983/1951] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:49 2022] DELETE /identity/v3/users/fd15e9fec2af4261990e3b6bcb5dea8c => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:49.709111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d336d66-cd34-44b5-bd76-814f3959a48d None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:49.709991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d336d66-cd34-44b5-bd76-814f3959a48d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:49.710277 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d336d66-cd34-44b5-bd76-814f3959a48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:49.728450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 969/1952] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:49.731143 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0d336d66-cd34-44b5-bd76-814f3959a48d None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:49.733043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0d336d66-cd34-44b5-bd76-814f3959a48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:49.739636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5251ae-44ac-40d6-a456-b911b2a998cb None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:49.740976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-7b5251ae-44ac-40d6-a456-b911b2a998cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:49.795893 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 984/1953] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:52:49.799217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5251ae-44ac-40d6-a456-b911b2a998cb None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:49.807691 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d336d66-cd34-44b5-bd76-814f3959a48d None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:49.854227 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45363a6-102c-4dbd-a4c9-ca986f99633c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:49.855030 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45363a6-102c-4dbd-a4c9-ca986f99633c service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:49.855382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45363a6-102c-4dbd-a4c9-ca986f99633c service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:49.855928 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45363a6-102c-4dbd-a4c9-ca986f99633c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:49.871480 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d336d66-cd34-44b5-bd76-814f3959a48d None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:49.871965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45363a6-102c-4dbd-a4c9-ca986f99633c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:49.875914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb94c3-52b6-4036-8f3a-d6c6d68ee7b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:49.876150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45363a6-102c-4dbd-a4c9-ca986f99633c service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:49.876555 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb94c3-52b6-4036-8f3a-d6c6d68ee7b4 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:49.876870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb94c3-52b6-4036-8f3a-d6c6d68ee7b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:49.877188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb94c3-52b6-4036-8f3a-d6c6d68ee7b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:49.884817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:49.888964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb94c3-52b6-4036-8f3a-d6c6d68ee7b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:49.890374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb94c3-52b6-4036-8f3a-d6c6d68ee7b4 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:49.896874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 970/1954] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:49.901776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 985/1955] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:49.976973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:50.377281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:50.492317 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45363a6-102c-4dbd-a4c9-ca986f99633c service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:50.493826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c45363a6-102c-4dbd-a4c9-ca986f99633c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:50.507386 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45363a6-102c-4dbd-a4c9-ca986f99633c service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:50.512927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:50.513593 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:50.513923 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:50.514225 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] PATH_INFO: `/v3/projects/cfedc6a60ab54b75bf9a619ed8369c23` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:50.515371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfedc6a60ab54b75bf9a619ed8369c23)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:50.517378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:50.553187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb94c3-52b6-4036-8f3a-d6c6d68ee7b4 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:50.554924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-61fb94c3-52b6-4036-8f3a-d6c6d68ee7b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:50.577297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:50.587023 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb94c3-52b6-4036-8f3a-d6c6d68ee7b4 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:50.597960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:50.599777 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:50.600123 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:50.600759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] PATH_INFO: `/v3/projects/fbc1edc7685f4b5e8b896244bb6b9479` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:50.602350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc1edc7685f4b5e8b896244bb6b9479)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:50.630841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:50.664179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfedc6a60ab54b75bf9a619ed8369c23', 'request_id': 'req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:50.673450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfedc6a60ab54b75bf9a619ed8369c23', 'request_id': 'req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:50.816197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfedc6a60ab54b75bf9a619ed8369c23', 'request_id': 'req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:50.833094 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] The token cache is being invalidate because project cfedc6a60ab54b75bf9a619ed8369c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:50.833470 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:50.834322 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:50.836242 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 971/1956] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:50 2022] DELETE /identity/v3/projects/cfedc6a60ab54b75bf9a619ed8369c23 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:50.840937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc1edc7685f4b5e8b896244bb6b9479', 'request_id': 'req-61a67931-1c85-47af-a540-7f39b1efb694'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:50.846494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:50.848408 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:50.863861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc1edc7685f4b5e8b896244bb6b9479', 'request_id': 'req-61a67931-1c85-47af-a540-7f39b1efb694'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:50.871043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc1edc7685f4b5e8b896244bb6b9479', 'request_id': 'req-61a67931-1c85-47af-a540-7f39b1efb694'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:50.938496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0a1ac-0aa9-4191-a06e-465abc5b7925 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:51.012688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] The token cache is being invalidate because project fbc1edc7685f4b5e8b896244bb6b9479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:51.012688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:51.012962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:51.014898 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 986/1957] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:50 2022] DELETE /identity/v3/projects/fbc1edc7685f4b5e8b896244bb6b9479 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:51.018967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:51.021155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:51.030111 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4117ceb-2866-4369-be77-a9112614a1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:51.030867 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4117ceb-2866-4369-be77-a9112614a1aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:51.031208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4117ceb-2866-4369-be77-a9112614a1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:51.032079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4117ceb-2866-4369-be77-a9112614a1aa admin admin] PATH_INFO: `/v3/users/663e632829ca46cf8bbeb70e2b05d1bb` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:51.032827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4117ceb-2866-4369-be77-a9112614a1aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=663e632829ca46cf8bbeb70e2b05d1bb)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:51.034950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4117ceb-2866-4369-be77-a9112614a1aa admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:51.127332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a67931-1c85-47af-a540-7f39b1efb694 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:51.138396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b9e1c-bd6f-445d-9164-0ad9075c723e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:51.139081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b9e1c-bd6f-445d-9164-0ad9075c723e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:51.139373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b9e1c-bd6f-445d-9164-0ad9075c723e admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:51.139652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b9e1c-bd6f-445d-9164-0ad9075c723e admin admin] PATH_INFO: `/v3/users/27ce124d578b47c1a723868899c1933b` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:51.141590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b9e1c-bd6f-445d-9164-0ad9075c723e admin admin] RBAC: Authorizing `identity:delete_user(user_id=27ce124d578b47c1a723868899c1933b)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:51.143864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b9e1c-bd6f-445d-9164-0ad9075c723e admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:51.257423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:51.262039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d4117ceb-2866-4369-be77-a9112614a1aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '663e632829ca46cf8bbeb70e2b05d1bb'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:51.298505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a70b9e1c-bd6f-445d-9164-0ad9075c723e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27ce124d578b47c1a723868899c1933b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:51.305288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a70b9e1c-bd6f-445d-9164-0ad9075c723e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27ce124d578b47c1a723868899c1933b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:51.319098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d4117ceb-2866-4369-be77-a9112614a1aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '663e632829ca46cf8bbeb70e2b05d1bb'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:51.320322 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d4117ceb-2866-4369-be77-a9112614a1aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '663e632829ca46cf8bbeb70e2b05d1bb'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:51.330620 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d4117ceb-2866-4369-be77-a9112614a1aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '663e632829ca46cf8bbeb70e2b05d1bb'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:51.340678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d4117ceb-2866-4369-be77-a9112614a1aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '663e632829ca46cf8bbeb70e2b05d1bb'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:51.346995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 972/1958] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:50 2022] DELETE /identity/v3/users/663e632829ca46cf8bbeb70e2b05d1bb => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:51.376605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a70b9e1c-bd6f-445d-9164-0ad9075c723e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27ce124d578b47c1a723868899c1933b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:51.382198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a70b9e1c-bd6f-445d-9164-0ad9075c723e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27ce124d578b47c1a723868899c1933b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:51.396971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a70b9e1c-bd6f-445d-9164-0ad9075c723e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27ce124d578b47c1a723868899c1933b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:51.404264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 987/1959] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:51 2022] DELETE /identity/v3/users/27ce124d578b47c1a723868899c1933b => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:51.680802 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:52.144112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4117ceb-2866-4369-be77-a9112614a1aa admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:52.146001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d4117ceb-2866-4369-be77-a9112614a1aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:52.213848 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b9e1c-bd6f-445d-9164-0ad9075c723e admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:52.216446 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a70b9e1c-bd6f-445d-9164-0ad9075c723e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:52.274908 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4117ceb-2866-4369-be77-a9112614a1aa admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:52.344933 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b9e1c-bd6f-445d-9164-0ad9075c723e admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:52.350750 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:52.357990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:52.359584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:52.360341 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:52.362402 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1 admin admin] PATH_INFO: `/v3/projects/1c0cfb77dd07465cb76d9619124d6d4a` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:52.365227 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c0cfb77dd07465cb76d9619124d6d4a)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:52.368331 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:52.390075 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca19de-9eeb-40e2-bfe0-7480431afa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:52.391623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca19de-9eeb-40e2-bfe0-7480431afa26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:52.392209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca19de-9eeb-40e2-bfe0-7480431afa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:52.393167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca19de-9eeb-40e2-bfe0-7480431afa26 admin admin] PATH_INFO: `/v3/projects/8d17b48ebb7c43548675a6f40a6c0ac6` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:52.395613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca19de-9eeb-40e2-bfe0-7480431afa26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d17b48ebb7c43548675a6f40a6c0ac6)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:52.403303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca19de-9eeb-40e2-bfe0-7480431afa26 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:52.515794 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c0cfb77dd07465cb76d9619124d6d4a', 'request_id': 'req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:52.532982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c0cfb77dd07465cb76d9619124d6d4a', 'request_id': 'req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:52.539253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e4ca19de-9eeb-40e2-bfe0-7480431afa26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d17b48ebb7c43548675a6f40a6c0ac6', 'request_id': 'req-e4ca19de-9eeb-40e2-bfe0-7480431afa26'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:52.547131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e4ca19de-9eeb-40e2-bfe0-7480431afa26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d17b48ebb7c43548675a6f40a6c0ac6', 'request_id': 'req-e4ca19de-9eeb-40e2-bfe0-7480431afa26'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:52.547356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c0cfb77dd07465cb76d9619124d6d4a', 'request_id': 'req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:52.618966 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1 admin admin] The token cache is being invalidate because project 1c0cfb77dd07465cb76d9619124d6d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:52.619681 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:52.620414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ee4b24d2-6112-4b3e-92d9-b7cf2f49b9d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:52.624060 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 988/1960] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:52 2022] DELETE /identity/v3/projects/1c0cfb77dd07465cb76d9619124d6d4a => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:52.632113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d57274-47ea-42e0-9e39-bdeaf43bdcae None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:52.632527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d57274-47ea-42e0-9e39-bdeaf43bdcae None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:52.632906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d57274-47ea-42e0-9e39-bdeaf43bdcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:52.663457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-89d57274-47ea-42e0-9e39-bdeaf43bdcae None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:52.665785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-89d57274-47ea-42e0-9e39-bdeaf43bdcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:52.686668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e4ca19de-9eeb-40e2-bfe0-7480431afa26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d17b48ebb7c43548675a6f40a6c0ac6', 'request_id': 'req-e4ca19de-9eeb-40e2-bfe0-7480431afa26'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:52.692710 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:52.704591 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e4ca19de-9eeb-40e2-bfe0-7480431afa26 admin admin] The token cache is being invalidate because project 8d17b48ebb7c43548675a6f40a6c0ac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:52.704998 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e4ca19de-9eeb-40e2-bfe0-7480431afa26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:52.707535 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e4ca19de-9eeb-40e2-bfe0-7480431afa26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:52.711300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 973/1961] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:52 2022] DELETE /identity/v3/projects/8d17b48ebb7c43548675a6f40a6c0ac6 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:52.719029 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb823116-db9d-46f0-a506-72bc88c95b0a None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:52.719313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb823116-db9d-46f0-a506-72bc88c95b0a None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:52.719595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb823116-db9d-46f0-a506-72bc88c95b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:52.773301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-bb823116-db9d-46f0-a506-72bc88c95b0a None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:52.775219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-bb823116-db9d-46f0-a506-72bc88c95b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:52.834520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 989/1962] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:52.836605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be78cf3-fb68-4282-ade2-2518b03037a6 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:52.837028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be78cf3-fb68-4282-ade2-2518b03037a6 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:52.837272 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be78cf3-fb68-4282-ade2-2518b03037a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:52.858394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 974/1963] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:52.868313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9d533-03fa-4553-b608-14bea393182a None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:52.868510 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9d533-03fa-4553-b608-14bea393182a None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:52.868850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9d533-03fa-4553-b608-14bea393182a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:52.878362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0be78cf3-fb68-4282-ade2-2518b03037a6 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:52.879841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0be78cf3-fb68-4282-ade2-2518b03037a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:52.893369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-c7e9d533-03fa-4553-b608-14bea393182a None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:52.898256 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 990/1964] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:52.898453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c7e9d533-03fa-4553-b608-14bea393182a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:52.900040 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be78cf3-fb68-4282-ade2-2518b03037a6 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:52.909473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 975/1965] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:52.914343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be78cf3-fb68-4282-ade2-2518b03037a6 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:52.914609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9d533-03fa-4553-b608-14bea393182a None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:52.920218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2d081-e930-4e08-8677-9cdc4ba5d16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:52.921077 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2d081-e930-4e08-8677-9cdc4ba5d16d admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:52.921374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2d081-e930-4e08-8677-9cdc4ba5d16d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:52.921653 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2d081-e930-4e08-8677-9cdc4ba5d16d admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:52.925171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2d081-e930-4e08-8677-9cdc4ba5d16d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:52.925171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2d081-e930-4e08-8677-9cdc4ba5d16d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:52.925171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2d081-e930-4e08-8677-9cdc4ba5d16d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:52.931921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9d533-03fa-4553-b608-14bea393182a None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:52.941672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 991/1966] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:52:52.944306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2d081-e930-4e08-8677-9cdc4ba5d16d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:52.952451 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c6215-dd7e-464e-92e0-7096a6f5157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:52.954599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c6215-dd7e-464e-92e0-7096a6f5157a admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:52.954599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c6215-dd7e-464e-92e0-7096a6f5157a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:52.954599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c6215-dd7e-464e-92e0-7096a6f5157a admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:52.954860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c6215-dd7e-464e-92e0-7096a6f5157a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:52:52.955919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c6215-dd7e-464e-92e0-7096a6f5157a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:52.957212 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c6215-dd7e-464e-92e0-7096a6f5157a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:52.960499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2d081-e930-4e08-8677-9cdc4ba5d16d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:52.970575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e09502-83c1-4865-aff3-5f1ab31a738f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:52.971535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e09502-83c1-4865-aff3-5f1ab31a738f admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:52.972092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e09502-83c1-4865-aff3-5f1ab31a738f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:52.972554 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e09502-83c1-4865-aff3-5f1ab31a738f admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:52.977745 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e09502-83c1-4865-aff3-5f1ab31a738f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:52.980356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e09502-83c1-4865-aff3-5f1ab31a738f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:52.999699 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 976/1967] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:52:53.002180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:53.004854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c6215-dd7e-464e-92e0-7096a6f5157a admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:53.017411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c6215-dd7e-464e-92e0-7096a6f5157a admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:53.024163 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e56b8c-9381-48a8-a979-e9383ed64591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:53.025623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e56b8c-9381-48a8-a979-e9383ed64591 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:53.026189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e56b8c-9381-48a8-a979-e9383ed64591 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:53.026746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e56b8c-9381-48a8-a979-e9383ed64591 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:53.027718 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e56b8c-9381-48a8-a979-e9383ed64591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:53.029864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e56b8c-9381-48a8-a979-e9383ed64591 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:53.097871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 992/1968] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:52 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:52:53.101767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e09502-83c1-4865-aff3-5f1ab31a738f admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:53.122657 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 977/1969] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:53 2022] POST /identity/v3/projects => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:53.127518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e56b8c-9381-48a8-a979-e9383ed64591 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:53.242636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e09502-83c1-4865-aff3-5f1ab31a738f admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:53.245382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e56b8c-9381-48a8-a979-e9383ed64591 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:53.249124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371a815-0651-4cba-9c01-14bf81e13fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:53.249984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371a815-0651-4cba-9c01-14bf81e13fea admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:53.250402 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371a815-0651-4cba-9c01-14bf81e13fea admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:53.250785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371a815-0651-4cba-9c01-14bf81e13fea admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:53.252043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371a815-0651-4cba-9c01-14bf81e13fea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:53.253420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50db924-9a60-41af-8d77-8e54fd67dfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:53.254399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50db924-9a60-41af-8d77-8e54fd67dfaa admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:53.254521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371a815-0651-4cba-9c01-14bf81e13fea admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:53.254692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50db924-9a60-41af-8d77-8e54fd67dfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:53.254972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50db924-9a60-41af-8d77-8e54fd67dfaa admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:53.255622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50db924-9a60-41af-8d77-8e54fd67dfaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:53.257632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50db924-9a60-41af-8d77-8e54fd67dfaa admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:53.402493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 993/1970] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:53 2022] POST /identity/v3/users => generated 286 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:52:53.406392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371a815-0651-4cba-9c01-14bf81e13fea admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:53.416707 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 978/1971] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:53 2022] POST /identity/v3/users => generated 284 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:52:53.419047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371a815-0651-4cba-9c01-14bf81e13fea admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:53.420924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50db924-9a60-41af-8d77-8e54fd67dfaa admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:53.425192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea335e-af58-4d29-96a7-33e09e9b3d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:53.425968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea335e-af58-4d29-96a7-33e09e9b3d4c admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:53.426294 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea335e-af58-4d29-96a7-33e09e9b3d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:53.426525 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea335e-af58-4d29-96a7-33e09e9b3d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:53.427167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea335e-af58-4d29-96a7-33e09e9b3d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:53.434748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea335e-af58-4d29-96a7-33e09e9b3d4c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:53.443800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50db924-9a60-41af-8d77-8e54fd67dfaa admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:53.448229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 994/1972] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:52:53.453044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea335e-af58-4d29-96a7-33e09e9b3d4c admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:53.453832 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53daf42-9fc3-4340-94a7-8be6ea45054b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:53.454818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53daf42-9fc3-4340-94a7-8be6ea45054b admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:53.455061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53daf42-9fc3-4340-94a7-8be6ea45054b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:53.455339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53daf42-9fc3-4340-94a7-8be6ea45054b admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:53.456050 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53daf42-9fc3-4340-94a7-8be6ea45054b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:53.457800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53daf42-9fc3-4340-94a7-8be6ea45054b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:53.465648 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea335e-af58-4d29-96a7-33e09e9b3d4c admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:53.484878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 979/1973] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:52:53.486962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53daf42-9fc3-4340-94a7-8be6ea45054b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:53.489476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe33f78-30fa-4642-9445-aa1830d8bebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:53.490340 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe33f78-30fa-4642-9445-aa1830d8bebb admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:53.490783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe33f78-30fa-4642-9445-aa1830d8bebb admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:53.491250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe33f78-30fa-4642-9445-aa1830d8bebb admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:53.491934 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe33f78-30fa-4642-9445-aa1830d8bebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:53.493928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe33f78-30fa-4642-9445-aa1830d8bebb admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:53.505666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53daf42-9fc3-4340-94a7-8be6ea45054b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:53.508787 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 995/1974] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:53.513307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe33f78-30fa-4642-9445-aa1830d8bebb admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:53.520072 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b86821-1da7-4880-b30b-9408cd6071b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:53.520315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:53.520972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b86821-1da7-4880-b30b-9408cd6071b0 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:53.521331 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b86821-1da7-4880-b30b-9408cd6071b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:53.521841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b86821-1da7-4880-b30b-9408cd6071b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:53.522749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b86821-1da7-4880-b30b-9408cd6071b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:53.524512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b86821-1da7-4880-b30b-9408cd6071b0 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:53.525590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe33f78-30fa-4642-9445-aa1830d8bebb admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:53.531852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448d8d6-eb93-4a25-a031-e2ed216b9e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:53.533834 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448d8d6-eb93-4a25-a031-e2ed216b9e00 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:53.534343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448d8d6-eb93-4a25-a031-e2ed216b9e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:53.534545 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448d8d6-eb93-4a25-a031-e2ed216b9e00 admin admin] PATH_INFO: `/v3/projects/bb370e5e106a4f73b4f9602e132d84d3/users/84db43dae1874ee4b7d2f7741bbfc360/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:53.549251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 980/1975] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:53.551548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b86821-1da7-4880-b30b-9408cd6071b0 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:53.561088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448d8d6-eb93-4a25-a031-e2ed216b9e00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb370e5e106a4f73b4f9602e132d84d3, user_id=84db43dae1874ee4b7d2f7741bbfc360, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:53.564203 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448d8d6-eb93-4a25-a031-e2ed216b9e00 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:53.575741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b86821-1da7-4880-b30b-9408cd6071b0 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:53.575741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f1f40-f27b-42ee-b47e-80fab86260d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:53.576823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1f40-f27b-42ee-b47e-80fab86260d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:53.576823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1f40-f27b-42ee-b47e-80fab86260d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:53.576823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1f40-f27b-42ee-b47e-80fab86260d0 admin admin] PATH_INFO: `/v3/projects/35160ff162a049869199c0f18de10f53/users/625c5e05f6494ee6912dc0741defc47e/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:53.594503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f1f40-f27b-42ee-b47e-80fab86260d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35160ff162a049869199c0f18de10f53, user_id=625c5e05f6494ee6912dc0741defc47e, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:53.598612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:53.599186 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f1f40-f27b-42ee-b47e-80fab86260d0 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:53.668837 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 981/1976] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:53 2022] PUT /identity/v3/projects/35160ff162a049869199c0f18de10f53/users/625c5e05f6494ee6912dc0741defc47e/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:53.678712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 996/1977] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:53 2022] PUT /identity/v3/projects/bb370e5e106a4f73b4f9602e132d84d3/users/84db43dae1874ee4b7d2f7741bbfc360/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:53.679629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881c931-b1cd-4d0f-8a6f-11736fc455af None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:53.679921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881c931-b1cd-4d0f-8a6f-11736fc455af None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:53.680235 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881c931-b1cd-4d0f-8a6f-11736fc455af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:53.693916 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e5cb0-7ffc-45fa-88da-fa075e1296d4 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:53.694612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e5cb0-7ffc-45fa-88da-fa075e1296d4 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:53.695092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e5cb0-7ffc-45fa-88da-fa075e1296d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:53.718366 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d881c931-b1cd-4d0f-8a6f-11736fc455af None None] MFA Rules not processed for user `625c5e05f6494ee6912dc0741defc47e`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:53.719688 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d881c931-b1cd-4d0f-8a6f-11736fc455af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:53.738831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-e72e5cb0-7ffc-45fa-88da-fa075e1296d4 None None] MFA Rules not processed for user `84db43dae1874ee4b7d2f7741bbfc360`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:53.743750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e72e5cb0-7ffc-45fa-88da-fa075e1296d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:53.798145 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 982/1978] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:53.802437 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881c931-b1cd-4d0f-8a6f-11736fc455af None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:53.815071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 997/1979] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:53.828926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e5cb0-7ffc-45fa-88da-fa075e1296d4 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:53.830575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e72e5cb0-7ffc-45fa-88da-fa075e1296d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:53.867774 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881c931-b1cd-4d0f-8a6f-11736fc455af None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:53.873557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33520-ba0f-442c-bee6-1e6b348972ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:53.874219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33520-ba0f-442c-bee6-1e6b348972ea admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:53.874506 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33520-ba0f-442c-bee6-1e6b348972ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:53.874784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33520-ba0f-442c-bee6-1e6b348972ea admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:53.875471 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f33520-ba0f-442c-bee6-1e6b348972ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:53.877845 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f33520-ba0f-442c-bee6-1e6b348972ea admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:53.902647 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e5cb0-7ffc-45fa-88da-fa075e1296d4 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:53.906757 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e310e4-cd10-414a-85fe-40964d76f67b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38de805ba5eb4aa58cf95d2082c2d216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:53.907392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e310e4-cd10-414a-85fe-40964d76f67b service glance] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:53.907669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e310e4-cd10-414a-85fe-40964d76f67b service glance] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:53.907936 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e310e4-cd10-414a-85fe-40964d76f67b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:53.917919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e310e4-cd10-414a-85fe-40964d76f67b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:53.919833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e310e4-cd10-414a-85fe-40964d76f67b service glance] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:53.929947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 998/1980] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:52:53.981194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 983/1981] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:52:53 2022] POST /identity/v3/projects => generated 369 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:52:53.985147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e310e4-cd10-414a-85fe-40964d76f67b service glance] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:54.002438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e310e4-cd10-414a-85fe-40964d76f67b service glance] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:54.078138 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889f022-941f-40be-8460-e9305fc56baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:54.078754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889f022-941f-40be-8460-e9305fc56baf admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:54.079041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889f022-941f-40be-8460-e9305fc56baf admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:54.079563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889f022-941f-40be-8460-e9305fc56baf admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:54.080201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1889f022-941f-40be-8460-e9305fc56baf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:54.082236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1889f022-941f-40be-8460-e9305fc56baf admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:54.222089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 999/1982] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:52:53 2022] POST /identity/v3/users => generated 291 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:52:54.226103 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33520-ba0f-442c-bee6-1e6b348972ea admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:54.236952 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33520-ba0f-442c-bee6-1e6b348972ea admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:54.241927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2dd627-670e-44ea-bd81-996d51fe818e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:54.242547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2dd627-670e-44ea-bd81-996d51fe818e admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:54.242830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2dd627-670e-44ea-bd81-996d51fe818e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:54.243103 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2dd627-670e-44ea-bd81-996d51fe818e admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:54.243869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2dd627-670e-44ea-bd81-996d51fe818e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:54.246424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2dd627-670e-44ea-bd81-996d51fe818e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:54.263667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 984/1983] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:54.264062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889f022-941f-40be-8460-e9305fc56baf admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:54.269663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:54.275108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889f022-941f-40be-8460-e9305fc56baf admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:54.280077 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33547ad6-7e4d-4018-b747-612694989405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:54.280776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33547ad6-7e4d-4018-b747-612694989405 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:54.281064 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33547ad6-7e4d-4018-b747-612694989405 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:54.281342 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33547ad6-7e4d-4018-b747-612694989405 admin admin] PATH_INFO: `/v3/projects/918fda275cbf42c69073a141888f78d0/users/222d928235354a6a917a5d328caeda5a/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:54.297004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33547ad6-7e4d-4018-b747-612694989405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=918fda275cbf42c69073a141888f78d0, user_id=222d928235354a6a917a5d328caeda5a, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:54.299797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33547ad6-7e4d-4018-b747-612694989405 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:54.441841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1000/1984] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:54 2022] PUT /identity/v3/projects/918fda275cbf42c69073a141888f78d0/users/222d928235354a6a917a5d328caeda5a/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:54.446461 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2dd627-670e-44ea-bd81-996d51fe818e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:54.466234 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2dd627-670e-44ea-bd81-996d51fe818e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:54.555825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:54.565831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f715bea-4a97-43ce-8c23-915b6ca4f6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:54.566462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f715bea-4a97-43ce-8c23-915b6ca4f6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:54.568149 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f715bea-4a97-43ce-8c23-915b6ca4f6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:54.568443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f715bea-4a97-43ce-8c23-915b6ca4f6e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:54.569346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f715bea-4a97-43ce-8c23-915b6ca4f6e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:54.572621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f715bea-4a97-43ce-8c23-915b6ca4f6e7 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:54.588672 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 985/1985] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:52:54.590373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33547ad6-7e4d-4018-b747-612694989405 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:54.609795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33547ad6-7e4d-4018-b747-612694989405 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:54.622127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95215f3-be95-42e2-ad3b-b9aaf0882935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:54.622807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95215f3-be95-42e2-ad3b-b9aaf0882935 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:54.623123 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95215f3-be95-42e2-ad3b-b9aaf0882935 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:54.623421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95215f3-be95-42e2-ad3b-b9aaf0882935 admin admin] PATH_INFO: `/v3/projects/918fda275cbf42c69073a141888f78d0/users/222d928235354a6a917a5d328caeda5a/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:54.625385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95215f3-be95-42e2-ad3b-b9aaf0882935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=918fda275cbf42c69073a141888f78d0, user_id=222d928235354a6a917a5d328caeda5a, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:54.628334 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95215f3-be95-42e2-ad3b-b9aaf0882935 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:54.641714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1001/1986] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:52:54 2022] PUT /identity/v3/projects/918fda275cbf42c69073a141888f78d0/users/222d928235354a6a917a5d328caeda5a/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:54.652880 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c59de-faea-4669-8993-2e8ea751239a None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:54.653179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c59de-faea-4669-8993-2e8ea751239a None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:54.653466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c59de-faea-4669-8993-2e8ea751239a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:54.700245 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-857c59de-faea-4669-8993-2e8ea751239a None None] MFA Rules not processed for user `222d928235354a6a917a5d328caeda5a`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:54.701545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-857c59de-faea-4669-8993-2e8ea751239a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:54.782301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 986/1987] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:54.793296 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95215f3-be95-42e2-ad3b-b9aaf0882935 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:54.795019 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a95215f3-be95-42e2-ad3b-b9aaf0882935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:54.859172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95215f3-be95-42e2-ad3b-b9aaf0882935 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:54.863486 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4cdc6-ac8e-4838-8255-df3f6690f480 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:54.864416 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4cdc6-ac8e-4838-8255-df3f6690f480 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:54.864831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4cdc6-ac8e-4838-8255-df3f6690f480 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:54.865204 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4cdc6-ac8e-4838-8255-df3f6690f480 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:54.878903 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf4cdc6-ac8e-4838-8255-df3f6690f480 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:54.880696 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf4cdc6-ac8e-4838-8255-df3f6690f480 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:54.893544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1002/1988] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:57.725078 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c59de-faea-4669-8993-2e8ea751239a None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:57.788205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c59de-faea-4669-8993-2e8ea751239a None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:57.792846 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80737902-e2cf-48f8-99e0-15811a07065c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:57.794720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80737902-e2cf-48f8-99e0-15811a07065c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:57.795002 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80737902-e2cf-48f8-99e0-15811a07065c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:57.795277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80737902-e2cf-48f8-99e0-15811a07065c admin admin] PATH_INFO: `/v3/users/625c5e05f6494ee6912dc0741defc47e` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:57.796403 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80737902-e2cf-48f8-99e0-15811a07065c admin admin] RBAC: Authorizing `identity:delete_user(user_id=625c5e05f6494ee6912dc0741defc47e)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:57.798480 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80737902-e2cf-48f8-99e0-15811a07065c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:58.050043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-80737902-e2cf-48f8-99e0-15811a07065c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '625c5e05f6494ee6912dc0741defc47e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:58.148504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-80737902-e2cf-48f8-99e0-15811a07065c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '625c5e05f6494ee6912dc0741defc47e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:58.149195 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-80737902-e2cf-48f8-99e0-15811a07065c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '625c5e05f6494ee6912dc0741defc47e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:58.161179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-80737902-e2cf-48f8-99e0-15811a07065c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '625c5e05f6494ee6912dc0741defc47e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:58.178980 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:58.192602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-80737902-e2cf-48f8-99e0-15811a07065c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '625c5e05f6494ee6912dc0741defc47e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:58.203735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 987/1989] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:57 2022] DELETE /identity/v3/users/625c5e05f6494ee6912dc0741defc47e => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:58.207617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd7c05-11fa-4d74-986e-650c258df2eb None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:58.208115 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd7c05-11fa-4d74-986e-650c258df2eb None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:58.208230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd7c05-11fa-4d74-986e-650c258df2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:58.236047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-d0bd7c05-11fa-4d74-986e-650c258df2eb None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:58.255372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d0bd7c05-11fa-4d74-986e-650c258df2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:58.303673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1003/1990] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:58.314122 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80737902-e2cf-48f8-99e0-15811a07065c admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:58.315621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-80737902-e2cf-48f8-99e0-15811a07065c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:58.376907 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80737902-e2cf-48f8-99e0-15811a07065c admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:58.380688 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a45b86-d6cd-4d28-95df-a56dd15fd839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:58.381293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a45b86-d6cd-4d28-95df-a56dd15fd839 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:58.381575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a45b86-d6cd-4d28-95df-a56dd15fd839 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:58.381851 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a45b86-d6cd-4d28-95df-a56dd15fd839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:58.391540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a45b86-d6cd-4d28-95df-a56dd15fd839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:58.393013 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a45b86-d6cd-4d28-95df-a56dd15fd839 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:58.402903 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 988/1991] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:58.823342 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd7c05-11fa-4d74-986e-650c258df2eb None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:58.824893 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d0bd7c05-11fa-4d74-986e-650c258df2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:58.837489 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd7c05-11fa-4d74-986e-650c258df2eb None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:58.842420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:58.843060 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:58.843339 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:58.843612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] PATH_INFO: `/v3/projects/35160ff162a049869199c0f18de10f53` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:58.844767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] RBAC: Authorizing `identity:delete_project(project_id=35160ff162a049869199c0f18de10f53)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:58.846686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:58.971106 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35160ff162a049869199c0f18de10f53', 'request_id': 'req-400874c8-4e1d-4527-a88e-413b7b4af76c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:58.977274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35160ff162a049869199c0f18de10f53', 'request_id': 'req-400874c8-4e1d-4527-a88e-413b7b4af76c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:58.983666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35160ff162a049869199c0f18de10f53', 'request_id': 'req-400874c8-4e1d-4527-a88e-413b7b4af76c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:58.985384 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:59.078985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] The token cache is being invalidate because project 35160ff162a049869199c0f18de10f53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:59.079468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:59.080108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:59.081850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1004/1992] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:58 2022] DELETE /identity/v3/projects/35160ff162a049869199c0f18de10f53 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:59.085810 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a45b86-d6cd-4d28-95df-a56dd15fd839 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:59.096455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a45b86-d6cd-4d28-95df-a56dd15fd839 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:59.174240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581c979-a53f-4bab-bfd0-45760da8b941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:59.174988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581c979-a53f-4bab-bfd0-45760da8b941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:59.175276 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581c979-a53f-4bab-bfd0-45760da8b941 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:59.175556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581c979-a53f-4bab-bfd0-45760da8b941 admin admin] PATH_INFO: `/v3/users/222d928235354a6a917a5d328caeda5a` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:59.176751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581c979-a53f-4bab-bfd0-45760da8b941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=222d928235354a6a917a5d328caeda5a)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:59.178716 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581c979-a53f-4bab-bfd0-45760da8b941 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:59.424456 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-3581c979-a53f-4bab-bfd0-45760da8b941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '222d928235354a6a917a5d328caeda5a'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:59.473691 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-3581c979-a53f-4bab-bfd0-45760da8b941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '222d928235354a6a917a5d328caeda5a'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:59.474595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-3581c979-a53f-4bab-bfd0-45760da8b941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '222d928235354a6a917a5d328caeda5a'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:59.482677 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-3581c979-a53f-4bab-bfd0-45760da8b941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '222d928235354a6a917a5d328caeda5a'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:59.498756 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-3581c979-a53f-4bab-bfd0-45760da8b941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '222d928235354a6a917a5d328caeda5a'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:59.504988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 989/1993] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:59 2022] DELETE /identity/v3/users/222d928235354a6a917a5d328caeda5a => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:59.962353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:59.963858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:59.981165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400874c8-4e1d-4527-a88e-413b7b4af76c admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:00.041699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:00.042487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:00.042683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:00.042963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a admin admin] PATH_INFO: `/v3/projects/918fda275cbf42c69073a141888f78d0` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:00.044089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a admin admin] RBAC: Authorizing `identity:delete_project(project_id=918fda275cbf42c69073a141888f78d0)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:00.046043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:00.171710 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '918fda275cbf42c69073a141888f78d0', 'request_id': 'req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:00.179120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '918fda275cbf42c69073a141888f78d0', 'request_id': 'req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:00.185439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '918fda275cbf42c69073a141888f78d0', 'request_id': 'req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:00.284761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a admin admin] The token cache is being invalidate because project 918fda275cbf42c69073a141888f78d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:00.285057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:00.285704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f1c574d7-a8d1-4ba2-a3d1-f03a100ea54a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:00.287549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1005/1994] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:59 2022] DELETE /identity/v3/projects/918fda275cbf42c69073a141888f78d0 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:00.292258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52203d-18bb-4e91-ab79-d5794d1f3955 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:00.292756 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52203d-18bb-4e91-ab79-d5794d1f3955 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:00.293445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52203d-18bb-4e91-ab79-d5794d1f3955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:00.313842 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-2a52203d-18bb-4e91-ab79-d5794d1f3955 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:00.315069 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2a52203d-18bb-4e91-ab79-d5794d1f3955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:00.332588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:00.380380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 990/1995] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:00.388222 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709c4ba-2a5f-4fbe-98a0-6f5c9d0f2536 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:00.388515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709c4ba-2a5f-4fbe-98a0-6f5c9d0f2536 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:00.388984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709c4ba-2a5f-4fbe-98a0-6f5c9d0f2536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:00.409307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-c709c4ba-2a5f-4fbe-98a0-6f5c9d0f2536 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:00.410552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c709c4ba-2a5f-4fbe-98a0-6f5c9d0f2536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:00.419202 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1006/1996] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:00.423528 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52203d-18bb-4e91-ab79-d5794d1f3955 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:00.433387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52203d-18bb-4e91-ab79-d5794d1f3955 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:00.439058 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d5a58-54fe-4233-a43a-97748e840d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:00.439661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d5a58-54fe-4233-a43a-97748e840d64 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:00.439967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d5a58-54fe-4233-a43a-97748e840d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:00.441083 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d5a58-54fe-4233-a43a-97748e840d64 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:00.441709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d5a58-54fe-4233-a43a-97748e840d64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:53:00.441994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d5a58-54fe-4233-a43a-97748e840d64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:00.443595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d5a58-54fe-4233-a43a-97748e840d64 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:00.456599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 991/1997] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:53:00.461065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709c4ba-2a5f-4fbe-98a0-6f5c9d0f2536 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:00.472172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709c4ba-2a5f-4fbe-98a0-6f5c9d0f2536 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:00.477200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0624f-fd55-41e1-851b-836dd6b75bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:00.477820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0624f-fd55-41e1-851b-836dd6b75bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:00.478105 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0624f-fd55-41e1-851b-836dd6b75bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:00.478372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0624f-fd55-41e1-851b-836dd6b75bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:00.479003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0624f-fd55-41e1-851b-836dd6b75bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:00.481563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0624f-fd55-41e1-851b-836dd6b75bb4 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:00.579778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1007/1998] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:00 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:00.583567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d5a58-54fe-4233-a43a-97748e840d64 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:00.596248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d5a58-54fe-4233-a43a-97748e840d64 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:00.659543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a46ea2-ef7d-4087-90a6-a35f63f09a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:00.660346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a46ea2-ef7d-4087-90a6-a35f63f09a78 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:00.660486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a46ea2-ef7d-4087-90a6-a35f63f09a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:00.660761 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a46ea2-ef7d-4087-90a6-a35f63f09a78 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:00.661557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a46ea2-ef7d-4087-90a6-a35f63f09a78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:00.663545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a46ea2-ef7d-4087-90a6-a35f63f09a78 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:00.754298 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 992/1999] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:00 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:00.758623 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0624f-fd55-41e1-851b-836dd6b75bb4 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:00.769308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0624f-fd55-41e1-851b-836dd6b75bb4 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:00.773960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e0324-f44d-400d-bc2d-b4bad205c448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:00.774558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e0324-f44d-400d-bc2d-b4bad205c448 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:00.774843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e0324-f44d-400d-bc2d-b4bad205c448 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:00.775120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e0324-f44d-400d-bc2d-b4bad205c448 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:00.775762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e0324-f44d-400d-bc2d-b4bad205c448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:00.777902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e0324-f44d-400d-bc2d-b4bad205c448 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:00.787884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1008/2000] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:00.796831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a46ea2-ef7d-4087-90a6-a35f63f09a78 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:00.806760 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a46ea2-ef7d-4087-90a6-a35f63f09a78 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:00.811545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f229474-a1f5-4ccf-b0db-cc3349119d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:00.812132 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f229474-a1f5-4ccf-b0db-cc3349119d1c admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:00.812412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f229474-a1f5-4ccf-b0db-cc3349119d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:00.812721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f229474-a1f5-4ccf-b0db-cc3349119d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:00.813355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f229474-a1f5-4ccf-b0db-cc3349119d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:00.815018 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f229474-a1f5-4ccf-b0db-cc3349119d1c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:00.826279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 993/2001] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:00.830920 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e0324-f44d-400d-bc2d-b4bad205c448 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:00.842705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e0324-f44d-400d-bc2d-b4bad205c448 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:00.851728 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c30099-6e4e-4f1f-ad4e-219a9c29b91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:00.851728 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c30099-6e4e-4f1f-ad4e-219a9c29b91b admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:00.851728 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c30099-6e4e-4f1f-ad4e-219a9c29b91b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:00.851728 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c30099-6e4e-4f1f-ad4e-219a9c29b91b admin admin] PATH_INFO: `/v3/projects/9ae92b611cd342049cf72e28c41e472d/users/19bb48502370411da34a5cf2b6099e36/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:00.864377 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c30099-6e4e-4f1f-ad4e-219a9c29b91b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ae92b611cd342049cf72e28c41e472d, user_id=19bb48502370411da34a5cf2b6099e36, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:00.867130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c30099-6e4e-4f1f-ad4e-219a9c29b91b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:00.947971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1009/2002] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:00 2022] PUT /identity/v3/projects/9ae92b611cd342049cf72e28c41e472d/users/19bb48502370411da34a5cf2b6099e36/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:00.965107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12472355-6ad9-4f79-8dc4-126429a02c1c None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:00.965292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12472355-6ad9-4f79-8dc4-126429a02c1c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:00.965501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12472355-6ad9-4f79-8dc4-126429a02c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:00.993771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-12472355-6ad9-4f79-8dc4-126429a02c1c None None] MFA Rules not processed for user `19bb48502370411da34a5cf2b6099e36`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:00.994994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-12472355-6ad9-4f79-8dc4-126429a02c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:01.061471 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 994/2003] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:01.066336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c30099-6e4e-4f1f-ad4e-219a9c29b91b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:01.088186 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:01.133709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c30099-6e4e-4f1f-ad4e-219a9c29b91b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:01.139072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933664da-525d-4d5a-a4e7-abdd6ad71fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:01.139686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933664da-525d-4d5a-a4e7-abdd6ad71fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:01.140005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933664da-525d-4d5a-a4e7-abdd6ad71fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:01.140366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933664da-525d-4d5a-a4e7-abdd6ad71fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:01.140982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933664da-525d-4d5a-a4e7-abdd6ad71fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:01.142929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933664da-525d-4d5a-a4e7-abdd6ad71fd2 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:01.236471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1010/2004] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:01 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:01.242235 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12472355-6ad9-4f79-8dc4-126429a02c1c None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:01.252615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12472355-6ad9-4f79-8dc4-126429a02c1c None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:01.263424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:01.325556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa9f37-9f9c-4b84-8672-b461cd42d805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:01.326175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa9f37-9f9c-4b84-8672-b461cd42d805 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:01.327300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa9f37-9f9c-4b84-8672-b461cd42d805 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:01.327300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa9f37-9f9c-4b84-8672-b461cd42d805 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:01.327537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa9f37-9f9c-4b84-8672-b461cd42d805 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:01.332989 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa9f37-9f9c-4b84-8672-b461cd42d805 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:01.537426 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 995/2005] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:01 2022] POST /identity/v3/users => generated 299 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:01.541008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933664da-525d-4d5a-a4e7-abdd6ad71fd2 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:01.553290 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933664da-525d-4d5a-a4e7-abdd6ad71fd2 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:01.558808 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f684c6-1a01-4561-b467-19725984b95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:01.559501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f684c6-1a01-4561-b467-19725984b95c admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:01.559790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f684c6-1a01-4561-b467-19725984b95c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:01.560072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f684c6-1a01-4561-b467-19725984b95c admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:01.562246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f684c6-1a01-4561-b467-19725984b95c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:01.562627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f684c6-1a01-4561-b467-19725984b95c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:01.573742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1011/2006] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:01.578939 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa9f37-9f9c-4b84-8672-b461cd42d805 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:01.589438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa9f37-9f9c-4b84-8672-b461cd42d805 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:01.595080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a63a4f-1932-4a26-aca7-69b2c8674532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:01.595752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a63a4f-1932-4a26-aca7-69b2c8674532 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:01.596052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a63a4f-1932-4a26-aca7-69b2c8674532 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:01.596328 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a63a4f-1932-4a26-aca7-69b2c8674532 admin admin] PATH_INFO: `/v3/projects/918c0e8dda8245d588d55916b3ee527e/users/004c2f15144542f7adf16554dbc5f7c9/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:01.612293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a63a4f-1932-4a26-aca7-69b2c8674532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=918c0e8dda8245d588d55916b3ee527e, user_id=004c2f15144542f7adf16554dbc5f7c9, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:01.615140 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a63a4f-1932-4a26-aca7-69b2c8674532 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:01.722624 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 996/2007] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:01 2022] PUT /identity/v3/projects/918c0e8dda8245d588d55916b3ee527e/users/004c2f15144542f7adf16554dbc5f7c9/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:01.727249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f684c6-1a01-4561-b467-19725984b95c admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:01.738155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f684c6-1a01-4561-b467-19725984b95c admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:01.838860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadafc2d-07e9-4067-90f7-165dfb1d0c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:01.843016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadafc2d-07e9-4067-90f7-165dfb1d0c53 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:01.843016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadafc2d-07e9-4067-90f7-165dfb1d0c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:01.843016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadafc2d-07e9-4067-90f7-165dfb1d0c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:01.843016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadafc2d-07e9-4067-90f7-165dfb1d0c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:01.843499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadafc2d-07e9-4067-90f7-165dfb1d0c53 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:01.857213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1012/2008] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:01.858966 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a63a4f-1932-4a26-aca7-69b2c8674532 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:01.872055 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a63a4f-1932-4a26-aca7-69b2c8674532 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:01.878390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d55ecc-daca-4a3f-b31c-6b0bd768663d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:01.879161 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d55ecc-daca-4a3f-b31c-6b0bd768663d admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:01.879756 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d55ecc-daca-4a3f-b31c-6b0bd768663d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:01.880114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d55ecc-daca-4a3f-b31c-6b0bd768663d admin admin] PATH_INFO: `/v3/projects/918c0e8dda8245d588d55916b3ee527e/users/004c2f15144542f7adf16554dbc5f7c9/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:01.882312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d55ecc-daca-4a3f-b31c-6b0bd768663d admin admin] RBAC: Authorizing `identity:create_grant(project_id=918c0e8dda8245d588d55916b3ee527e, user_id=004c2f15144542f7adf16554dbc5f7c9, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:01.885543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d55ecc-daca-4a3f-b31c-6b0bd768663d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:01.901400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 997/2009] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:01 2022] PUT /identity/v3/projects/918c0e8dda8245d588d55916b3ee527e/users/004c2f15144542f7adf16554dbc5f7c9/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:01.913464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed29f2-8a3c-4f11-bcc1-1a30665f243e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:01.913854 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed29f2-8a3c-4f11-bcc1-1a30665f243e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:01.914210 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed29f2-8a3c-4f11-bcc1-1a30665f243e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:01.951897 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-21ed29f2-8a3c-4f11-bcc1-1a30665f243e None None] MFA Rules not processed for user `004c2f15144542f7adf16554dbc5f7c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:01.953464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-21ed29f2-8a3c-4f11-bcc1-1a30665f243e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:02.048236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1013/2010] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:02.059539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d55ecc-daca-4a3f-b31c-6b0bd768663d admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:02.061267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-86d55ecc-daca-4a3f-b31c-6b0bd768663d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:02.118422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d55ecc-daca-4a3f-b31c-6b0bd768663d admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:02.122771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d3d9f-a93b-49e6-baae-a1055151e2b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:02.123377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d3d9f-a93b-49e6-baae-a1055151e2b9 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:02.123663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d3d9f-a93b-49e6-baae-a1055151e2b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:02.123944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d3d9f-a93b-49e6-baae-a1055151e2b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:02.133154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d3d9f-a93b-49e6-baae-a1055151e2b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:02.134614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d3d9f-a93b-49e6-baae-a1055151e2b9 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:02.144802 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 998/2011] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:02 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:03.205361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed29f2-8a3c-4f11-bcc1-1a30665f243e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:03.219018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:03.270263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed29f2-8a3c-4f11-bcc1-1a30665f243e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:03.275071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d8a4c-d716-49da-9e1f-c1f054be6660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:03.275787 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8a4c-d716-49da-9e1f-c1f054be6660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:03.276013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8a4c-d716-49da-9e1f-c1f054be6660 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:03.276286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8a4c-d716-49da-9e1f-c1f054be6660 admin admin] PATH_INFO: `/v3/users/19bb48502370411da34a5cf2b6099e36` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:03.277468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d8a4c-d716-49da-9e1f-c1f054be6660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19bb48502370411da34a5cf2b6099e36)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:03.280195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d8a4c-d716-49da-9e1f-c1f054be6660 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:03.487533 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-526d8a4c-d716-49da-9e1f-c1f054be6660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19bb48502370411da34a5cf2b6099e36'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:03.494365 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-526d8a4c-d716-49da-9e1f-c1f054be6660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19bb48502370411da34a5cf2b6099e36'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:03.553090 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-526d8a4c-d716-49da-9e1f-c1f054be6660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19bb48502370411da34a5cf2b6099e36'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:03.553717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-526d8a4c-d716-49da-9e1f-c1f054be6660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19bb48502370411da34a5cf2b6099e36'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:03.566779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-526d8a4c-d716-49da-9e1f-c1f054be6660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19bb48502370411da34a5cf2b6099e36'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:03.572737 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1014/2012] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:03 2022] DELETE /identity/v3/users/19bb48502370411da34a5cf2b6099e36 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:03.577920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e58f0-50b5-4077-92b9-b4e8545be890 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:03.578466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e58f0-50b5-4077-92b9-b4e8545be890 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:03.578466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e58f0-50b5-4077-92b9-b4e8545be890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:03.597147 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-9f1e58f0-50b5-4077-92b9-b4e8545be890 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:03.598366 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9f1e58f0-50b5-4077-92b9-b4e8545be890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:03.658335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 999/2013] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:03.668195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d8a4c-d716-49da-9e1f-c1f054be6660 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:03.669704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-526d8a4c-d716-49da-9e1f-c1f054be6660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:03.731125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d8a4c-d716-49da-9e1f-c1f054be6660 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:03.734902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd4378-f97c-4422-bfb4-c99584f2a810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:03.735572 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd4378-f97c-4422-bfb4-c99584f2a810 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:03.735846 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd4378-f97c-4422-bfb4-c99584f2a810 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:03.736144 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd4378-f97c-4422-bfb4-c99584f2a810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:03.745008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd4378-f97c-4422-bfb4-c99584f2a810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:03.746472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd4378-f97c-4422-bfb4-c99584f2a810 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:03.756692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1015/2014] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:04.173606 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e58f0-50b5-4077-92b9-b4e8545be890 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:04.175100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9f1e58f0-50b5-4077-92b9-b4e8545be890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:04.190444 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e58f0-50b5-4077-92b9-b4e8545be890 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:04.195629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:04.200418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:04.200418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:04.200418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] PATH_INFO: `/v3/projects/9ae92b611cd342049cf72e28c41e472d` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:04.200418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ae92b611cd342049cf72e28c41e472d)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:04.200418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:04.203198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:04.315809 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ae92b611cd342049cf72e28c41e472d', 'request_id': 'req-5059d089-664a-4f9f-a97b-c92f0484ccd1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:04.322583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ae92b611cd342049cf72e28c41e472d', 'request_id': 'req-5059d089-664a-4f9f-a97b-c92f0484ccd1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:04.393240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ae92b611cd342049cf72e28c41e472d', 'request_id': 'req-5059d089-664a-4f9f-a97b-c92f0484ccd1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:04.406600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] The token cache is being invalidate because project 9ae92b611cd342049cf72e28c41e472d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:04.406989 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:04.407753 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:04.409595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1000/2015] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:04 2022] DELETE /identity/v3/projects/9ae92b611cd342049cf72e28c41e472d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:04.414788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd4378-f97c-4422-bfb4-c99584f2a810 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:04.426878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd4378-f97c-4422-bfb4-c99584f2a810 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:04.461008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:04.485792 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1982720-3a7e-4e1f-aed3-41a43fe64c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:04.486418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1982720-3a7e-4e1f-aed3-41a43fe64c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:04.486697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1982720-3a7e-4e1f-aed3-41a43fe64c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:04.486973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1982720-3a7e-4e1f-aed3-41a43fe64c55 admin admin] PATH_INFO: `/v3/users/004c2f15144542f7adf16554dbc5f7c9` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:04.488194 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1982720-3a7e-4e1f-aed3-41a43fe64c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=004c2f15144542f7adf16554dbc5f7c9)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:04.490211 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1982720-3a7e-4e1f-aed3-41a43fe64c55 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:04.818617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f1982720-3a7e-4e1f-aed3-41a43fe64c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '004c2f15144542f7adf16554dbc5f7c9'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:04.826245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f1982720-3a7e-4e1f-aed3-41a43fe64c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '004c2f15144542f7adf16554dbc5f7c9'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:04.993521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f1982720-3a7e-4e1f-aed3-41a43fe64c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '004c2f15144542f7adf16554dbc5f7c9'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:04.996910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f1982720-3a7e-4e1f-aed3-41a43fe64c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '004c2f15144542f7adf16554dbc5f7c9'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:05.006918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f1982720-3a7e-4e1f-aed3-41a43fe64c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '004c2f15144542f7adf16554dbc5f7c9'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:05.013037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1016/2016] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:04 2022] DELETE /identity/v3/users/004c2f15144542f7adf16554dbc5f7c9 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:05.526001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:05.527463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:05.543647 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059d089-664a-4f9f-a97b-c92f0484ccd1 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:05.621985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:05.622559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:05.623790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:05.624067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7 admin admin] PATH_INFO: `/v3/projects/918c0e8dda8245d588d55916b3ee527e` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:05.625237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=918c0e8dda8245d588d55916b3ee527e)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:05.627152 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:05.729983 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '918c0e8dda8245d588d55916b3ee527e', 'request_id': 'req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:05.737974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '918c0e8dda8245d588d55916b3ee527e', 'request_id': 'req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:05.827526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '918c0e8dda8245d588d55916b3ee527e', 'request_id': 'req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:05.839772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7 admin admin] The token cache is being invalidate because project 918c0e8dda8245d588d55916b3ee527e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:05.842521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:05.842521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1b520e66-bbcb-4a8c-84c1-2611fa0ea5f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:05.843356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1001/2017] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:05 2022] DELETE /identity/v3/projects/918c0e8dda8245d588d55916b3ee527e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:05.848443 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7f04a-1550-43f4-9e39-8a91c0b4f375 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:05.848946 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7f04a-1550-43f4-9e39-8a91c0b4f375 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:05.849142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7f04a-1550-43f4-9e39-8a91c0b4f375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:05.869183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-8ca7f04a-1550-43f4-9e39-8a91c0b4f375 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:05.870395 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8ca7f04a-1550-43f4-9e39-8a91c0b4f375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:05.928844 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1017/2018] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:05.937833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b36ed-cba4-4f3a-aad0-29d63caacbe4 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:05.938143 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b36ed-cba4-4f3a-aad0-29d63caacbe4 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:05.938426 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b36ed-cba4-4f3a-aad0-29d63caacbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:05.959579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-6d7b36ed-cba4-4f3a-aad0-29d63caacbe4 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:05.962195 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6d7b36ed-cba4-4f3a-aad0-29d63caacbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:05.975713 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1002/2019] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:05.981449 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7f04a-1550-43f4-9e39-8a91c0b4f375 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:05.998135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7f04a-1550-43f4-9e39-8a91c0b4f375 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:05.998135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54b088-7def-4578-bd11-d9c8dd053cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:05.998369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54b088-7def-4578-bd11-d9c8dd053cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:05.998653 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54b088-7def-4578-bd11-d9c8dd053cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:05.998928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54b088-7def-4578-bd11-d9c8dd053cc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:05.999651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54b088-7def-4578-bd11-d9c8dd053cc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:53:05.999957 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54b088-7def-4578-bd11-d9c8dd053cc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:06.001599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54b088-7def-4578-bd11-d9c8dd053cc6 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:06.014100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1018/2020] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:53:06.018555 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b36ed-cba4-4f3a-aad0-29d63caacbe4 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:06.028018 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b36ed-cba4-4f3a-aad0-29d63caacbe4 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:06.034221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b5a0b-4341-4afb-a508-3402292c8f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:06.034982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5a0b-4341-4afb-a508-3402292c8f56 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:06.035271 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5a0b-4341-4afb-a508-3402292c8f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:06.035596 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5a0b-4341-4afb-a508-3402292c8f56 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:06.040685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b5a0b-4341-4afb-a508-3402292c8f56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:06.040685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b5a0b-4341-4afb-a508-3402292c8f56 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:06.054705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:06.109368 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1003/2021] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:06 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:06.113688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54b088-7def-4578-bd11-d9c8dd053cc6 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:06.124782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54b088-7def-4578-bd11-d9c8dd053cc6 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:06.169259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:06.186554 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc2f43-1f81-4698-a737-a61201ee1e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:06.186788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc2f43-1f81-4698-a737-a61201ee1e36 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:06.187023 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc2f43-1f81-4698-a737-a61201ee1e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:06.187300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc2f43-1f81-4698-a737-a61201ee1e36 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:06.187938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc2f43-1f81-4698-a737-a61201ee1e36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:06.189898 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc2f43-1f81-4698-a737-a61201ee1e36 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:06.299236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1019/2022] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:06 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:06.307672 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b5a0b-4341-4afb-a508-3402292c8f56 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:06.318002 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b5a0b-4341-4afb-a508-3402292c8f56 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:06.322545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf3571-3026-4dcd-9aa6-eb7a3038c1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:06.323265 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf3571-3026-4dcd-9aa6-eb7a3038c1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:06.323560 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf3571-3026-4dcd-9aa6-eb7a3038c1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:06.323822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf3571-3026-4dcd-9aa6-eb7a3038c1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:06.324459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf3571-3026-4dcd-9aa6-eb7a3038c1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:06.326621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf3571-3026-4dcd-9aa6-eb7a3038c1d7 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:06.339178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1004/2023] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:06.342061 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc2f43-1f81-4698-a737-a61201ee1e36 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:06.354999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc2f43-1f81-4698-a737-a61201ee1e36 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:06.362717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71950c60-10b5-4a7e-8189-06bf132a985c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:06.364714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950c60-10b5-4a7e-8189-06bf132a985c admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:06.364714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950c60-10b5-4a7e-8189-06bf132a985c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:06.364714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950c60-10b5-4a7e-8189-06bf132a985c admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:06.364714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71950c60-10b5-4a7e-8189-06bf132a985c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:06.366179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71950c60-10b5-4a7e-8189-06bf132a985c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:06.382627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1020/2024] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:06.387668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf3571-3026-4dcd-9aa6-eb7a3038c1d7 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:06.401623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf3571-3026-4dcd-9aa6-eb7a3038c1d7 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:06.409442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa70ca2-84f7-426b-9b35-4f21df16e1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:06.410431 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa70ca2-84f7-426b-9b35-4f21df16e1d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:06.410817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa70ca2-84f7-426b-9b35-4f21df16e1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:06.411198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa70ca2-84f7-426b-9b35-4f21df16e1d2 admin admin] PATH_INFO: `/v3/projects/5ab57f71ced046a3bc9fd451d23e9af8/users/97c18d4518fa4e3abaac5bd9bc1fd02e/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:06.427527 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa70ca2-84f7-426b-9b35-4f21df16e1d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab57f71ced046a3bc9fd451d23e9af8, user_id=97c18d4518fa4e3abaac5bd9bc1fd02e, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:06.430336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa70ca2-84f7-426b-9b35-4f21df16e1d2 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:06.502266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1005/2025] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:06 2022] PUT /identity/v3/projects/5ab57f71ced046a3bc9fd451d23e9af8/users/97c18d4518fa4e3abaac5bd9bc1fd02e/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:06.515935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44021d1f-3fbc-4ff9-878a-9414dca98e4a None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:06.515935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44021d1f-3fbc-4ff9-878a-9414dca98e4a None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:06.515935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44021d1f-3fbc-4ff9-878a-9414dca98e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:06.557478 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-44021d1f-3fbc-4ff9-878a-9414dca98e4a None None] MFA Rules not processed for user `97c18d4518fa4e3abaac5bd9bc1fd02e`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:06.560041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-44021d1f-3fbc-4ff9-878a-9414dca98e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:06.652337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1021/2026] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:06.653460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa70ca2-84f7-426b-9b35-4f21df16e1d2 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:06.762083 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa70ca2-84f7-426b-9b35-4f21df16e1d2 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:06.768455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67863308-f175-4302-8d26-851db0c07a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:06.769683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67863308-f175-4302-8d26-851db0c07a27 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:06.770097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67863308-f175-4302-8d26-851db0c07a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:06.770585 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67863308-f175-4302-8d26-851db0c07a27 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:06.771407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67863308-f175-4302-8d26-851db0c07a27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:06.774407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67863308-f175-4302-8d26-851db0c07a27 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:06.874174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1006/2027] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:06 2022] POST /identity/v3/projects => generated 383 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:06.875148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44021d1f-3fbc-4ff9-878a-9414dca98e4a None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:06.886041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44021d1f-3fbc-4ff9-878a-9414dca98e4a None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:06.981219 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ca0c0-2e2c-454a-8bbd-e9ab653f0f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:06.983825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ca0c0-2e2c-454a-8bbd-e9ab653f0f2a admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:06.983825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ca0c0-2e2c-454a-8bbd-e9ab653f0f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:06.983825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ca0c0-2e2c-454a-8bbd-e9ab653f0f2a admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:06.984303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ca0c0-2e2c-454a-8bbd-e9ab653f0f2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:06.986408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ca0c0-2e2c-454a-8bbd-e9ab653f0f2a admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:07.046945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1022/2028] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:06 2022] POST /identity/v3/users => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:53:07.051951 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67863308-f175-4302-8d26-851db0c07a27 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:07.065505 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67863308-f175-4302-8d26-851db0c07a27 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:07.070986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713044c-c2a9-4c02-95e4-7094b11dec70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:07.071608 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713044c-c2a9-4c02-95e4-7094b11dec70 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:07.071891 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713044c-c2a9-4c02-95e4-7094b11dec70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:07.072215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713044c-c2a9-4c02-95e4-7094b11dec70 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:07.072910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713044c-c2a9-4c02-95e4-7094b11dec70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:07.074565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713044c-c2a9-4c02-95e4-7094b11dec70 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:07.092771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1007/2029] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:07.096957 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ca0c0-2e2c-454a-8bbd-e9ab653f0f2a admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:07.109447 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ca0c0-2e2c-454a-8bbd-e9ab653f0f2a admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:07.114195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101ed4-a7bc-4d01-bbcb-a38fd8170302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:07.114865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101ed4-a7bc-4d01-bbcb-a38fd8170302 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:07.115155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101ed4-a7bc-4d01-bbcb-a38fd8170302 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:07.115437 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101ed4-a7bc-4d01-bbcb-a38fd8170302 admin admin] PATH_INFO: `/v3/projects/94842f2f01954064991f6bbeb381b94f/users/a3c4f2d854d24e2489cf774087d3d313/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:07.134545 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d101ed4-a7bc-4d01-bbcb-a38fd8170302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94842f2f01954064991f6bbeb381b94f, user_id=a3c4f2d854d24e2489cf774087d3d313, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:07.138050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d101ed4-a7bc-4d01-bbcb-a38fd8170302 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:07.199091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1023/2030] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:07 2022] PUT /identity/v3/projects/94842f2f01954064991f6bbeb381b94f/users/a3c4f2d854d24e2489cf774087d3d313/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:07.203664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713044c-c2a9-4c02-95e4-7094b11dec70 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:07.215652 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713044c-c2a9-4c02-95e4-7094b11dec70 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:07.285550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11e2f5-f750-4891-956c-f563dcb46f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:07.286182 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11e2f5-f750-4891-956c-f563dcb46f80 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:07.286525 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11e2f5-f750-4891-956c-f563dcb46f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:07.286934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11e2f5-f750-4891-956c-f563dcb46f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:07.287656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a11e2f5-f750-4891-956c-f563dcb46f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:07.289360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a11e2f5-f750-4891-956c-f563dcb46f80 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:07.305702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1008/2031] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:07.310158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101ed4-a7bc-4d01-bbcb-a38fd8170302 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:07.321877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101ed4-a7bc-4d01-bbcb-a38fd8170302 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:07.327471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd784d-55b5-4468-8b9e-2b71387f5f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:07.328100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd784d-55b5-4468-8b9e-2b71387f5f52 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:07.328386 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd784d-55b5-4468-8b9e-2b71387f5f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:07.328692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd784d-55b5-4468-8b9e-2b71387f5f52 admin admin] PATH_INFO: `/v3/projects/94842f2f01954064991f6bbeb381b94f/users/a3c4f2d854d24e2489cf774087d3d313/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:07.330876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd784d-55b5-4468-8b9e-2b71387f5f52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94842f2f01954064991f6bbeb381b94f, user_id=a3c4f2d854d24e2489cf774087d3d313, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:07.333839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd784d-55b5-4468-8b9e-2b71387f5f52 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:07.345916 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1024/2032] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:07 2022] PUT /identity/v3/projects/94842f2f01954064991f6bbeb381b94f/users/a3c4f2d854d24e2489cf774087d3d313/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:07.356135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fe656-0df7-4626-8682-281e71d3810d None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:07.356434 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fe656-0df7-4626-8682-281e71d3810d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:07.356785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fe656-0df7-4626-8682-281e71d3810d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:07.386501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-eb2fe656-0df7-4626-8682-281e71d3810d None None] MFA Rules not processed for user `a3c4f2d854d24e2489cf774087d3d313`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:07.387722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-eb2fe656-0df7-4626-8682-281e71d3810d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:07.405129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:07.456820 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1009/2033] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:07.681276 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd784d-55b5-4468-8b9e-2b71387f5f52 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:07.683138 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9cbd784d-55b5-4468-8b9e-2b71387f5f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:07.689146 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:07.750441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd784d-55b5-4468-8b9e-2b71387f5f52 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:07.753274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bf740-8b2f-4002-a2a7-54cfb0e6337f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:07.754153 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bf740-8b2f-4002-a2a7-54cfb0e6337f service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:07.754593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bf740-8b2f-4002-a2a7-54cfb0e6337f service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:07.755013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bf740-8b2f-4002-a2a7-54cfb0e6337f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:07.867628 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bf740-8b2f-4002-a2a7-54cfb0e6337f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:07.869154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bf740-8b2f-4002-a2a7-54cfb0e6337f service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:07.879463 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1025/2034] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:09.178512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2fe656-0df7-4626-8682-281e71d3810d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:09.179747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-eb2fe656-0df7-4626-8682-281e71d3810d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:09.246111 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2fe656-0df7-4626-8682-281e71d3810d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:09.250923 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff3317-870e-4e6f-a4eb-dff46d0efcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:09.251581 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff3317-870e-4e6f-a4eb-dff46d0efcfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:09.251861 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff3317-870e-4e6f-a4eb-dff46d0efcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:09.252137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff3317-870e-4e6f-a4eb-dff46d0efcfd admin admin] PATH_INFO: `/v3/users/8b1479e196844d6cba238f34924068d4` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:09.253338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff3317-870e-4e6f-a4eb-dff46d0efcfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b1479e196844d6cba238f34924068d4)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:09.255292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff3317-870e-4e6f-a4eb-dff46d0efcfd admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:09.454934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4bff3317-870e-4e6f-a4eb-dff46d0efcfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b1479e196844d6cba238f34924068d4'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:09.496499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4bff3317-870e-4e6f-a4eb-dff46d0efcfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b1479e196844d6cba238f34924068d4'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:09.497160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4bff3317-870e-4e6f-a4eb-dff46d0efcfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b1479e196844d6cba238f34924068d4'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:09.505352 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4bff3317-870e-4e6f-a4eb-dff46d0efcfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b1479e196844d6cba238f34924068d4'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:09.519196 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4bff3317-870e-4e6f-a4eb-dff46d0efcfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b1479e196844d6cba238f34924068d4'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:09.525377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1010/2035] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:09 2022] DELETE /identity/v3/users/8b1479e196844d6cba238f34924068d4 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:09.532979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f30b9-292a-438d-a951-1b8d12ff20af None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:09.532979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f30b9-292a-438d-a951-1b8d12ff20af None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:09.532979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f30b9-292a-438d-a951-1b8d12ff20af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:09.550147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-978f30b9-292a-438d-a951-1b8d12ff20af None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:09.551469 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-978f30b9-292a-438d-a951-1b8d12ff20af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:09.619232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1026/2036] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:09.629964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff3317-870e-4e6f-a4eb-dff46d0efcfd admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:09.631382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4bff3317-870e-4e6f-a4eb-dff46d0efcfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:09.667160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:09.688745 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff3317-870e-4e6f-a4eb-dff46d0efcfd admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:09.693038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c61ed-60c8-4674-9484-03b9160b6437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:09.693641 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c61ed-60c8-4674-9484-03b9160b6437 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:09.693925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c61ed-60c8-4674-9484-03b9160b6437 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:09.694225 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c61ed-60c8-4674-9484-03b9160b6437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:09.703920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c61ed-60c8-4674-9484-03b9160b6437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:09.705680 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c61ed-60c8-4674-9484-03b9160b6437 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:09.715031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1011/2037] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:10.156419 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f30b9-292a-438d-a951-1b8d12ff20af None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:10.157159 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-978f30b9-292a-438d-a951-1b8d12ff20af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:10.169721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f30b9-292a-438d-a951-1b8d12ff20af None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:10.183574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:10.184382 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:10.184866 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:10.185149 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51 admin admin] PATH_INFO: `/v3/projects/ce2ccfbe0e5141d0be1c854bb7be312d` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:10.186404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce2ccfbe0e5141d0be1c854bb7be312d)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:10.188836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:10.333227 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce2ccfbe0e5141d0be1c854bb7be312d', 'request_id': 'req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:10.336484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:10.344057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce2ccfbe0e5141d0be1c854bb7be312d', 'request_id': 'req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:10.351624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce2ccfbe0e5141d0be1c854bb7be312d', 'request_id': 'req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:10.428613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51 admin admin] The token cache is being invalidate because project ce2ccfbe0e5141d0be1c854bb7be312d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:10.428948 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:10.429965 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5e3f4112-4fc8-42d8-a022-5c99f6a87f51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:10.431327 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1027/2038] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:10 2022] DELETE /identity/v3/projects/ce2ccfbe0e5141d0be1c854bb7be312d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:10.436097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba1d2d-1c4b-4ef6-a441-a3d5874b7e04 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:10.436621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba1d2d-1c4b-4ef6-a441-a3d5874b7e04 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:10.436621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba1d2d-1c4b-4ef6-a441-a3d5874b7e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:10.476600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-7fba1d2d-1c4b-4ef6-a441-a3d5874b7e04 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:10.478197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-7fba1d2d-1c4b-4ef6-a441-a3d5874b7e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:10.573536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1012/2039] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:10.579594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad8cae-332b-431c-890e-e1c2fc550baa None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:10.580512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad8cae-332b-431c-890e-e1c2fc550baa None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:10.580971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad8cae-332b-431c-890e-e1c2fc550baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:10.606662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-d0ad8cae-332b-431c-890e-e1c2fc550baa None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:10.608082 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d0ad8cae-332b-431c-890e-e1c2fc550baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:10.617740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1028/2040] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:53:10.622463 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba1d2d-1c4b-4ef6-a441-a3d5874b7e04 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:10.632937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba1d2d-1c4b-4ef6-a441-a3d5874b7e04 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:10.638880 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304d3f5-0f2d-4f78-8ff0-555986c4afab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:10.643207 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304d3f5-0f2d-4f78-8ff0-555986c4afab admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:10.643207 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304d3f5-0f2d-4f78-8ff0-555986c4afab admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:10.643207 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304d3f5-0f2d-4f78-8ff0-555986c4afab admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:10.643207 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304d3f5-0f2d-4f78-8ff0-555986c4afab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:53:10.643207 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304d3f5-0f2d-4f78-8ff0-555986c4afab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:10.643771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304d3f5-0f2d-4f78-8ff0-555986c4afab admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:10.657903 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1013/2041] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:53:10.663085 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad8cae-332b-431c-890e-e1c2fc550baa None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:10.673774 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad8cae-332b-431c-890e-e1c2fc550baa None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:10.679037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53874d3d-0e5a-4406-8ceb-6d38bc73de56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:10.680137 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53874d3d-0e5a-4406-8ceb-6d38bc73de56 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:10.680719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53874d3d-0e5a-4406-8ceb-6d38bc73de56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:10.681094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53874d3d-0e5a-4406-8ceb-6d38bc73de56 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:10.681866 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53874d3d-0e5a-4406-8ceb-6d38bc73de56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:10.684030 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53874d3d-0e5a-4406-8ceb-6d38bc73de56 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:10.807606 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1029/2042] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:10 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:10.812064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304d3f5-0f2d-4f78-8ff0-555986c4afab admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:10.822694 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304d3f5-0f2d-4f78-8ff0-555986c4afab admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:10.890265 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd913e-8d80-4020-8cdd-9e13deaef919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:10.891625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd913e-8d80-4020-8cdd-9e13deaef919 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:10.892082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd913e-8d80-4020-8cdd-9e13deaef919 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:10.892361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd913e-8d80-4020-8cdd-9e13deaef919 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:10.893083 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd913e-8d80-4020-8cdd-9e13deaef919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:10.895169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd913e-8d80-4020-8cdd-9e13deaef919 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:10.980243 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1014/2043] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:10 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:10.981573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53874d3d-0e5a-4406-8ceb-6d38bc73de56 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:10.993171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53874d3d-0e5a-4406-8ceb-6d38bc73de56 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:10.998114 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3834c-7e8e-4b5e-a416-8265881be2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:10.999381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3834c-7e8e-4b5e-a416-8265881be2e8 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:10.999593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3834c-7e8e-4b5e-a416-8265881be2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:11.000362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3834c-7e8e-4b5e-a416-8265881be2e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:11.001793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3834c-7e8e-4b5e-a416-8265881be2e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:11.003922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3834c-7e8e-4b5e-a416-8265881be2e8 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:11.017755 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1030/2044] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:53:11.022576 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd913e-8d80-4020-8cdd-9e13deaef919 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:11.041654 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd913e-8d80-4020-8cdd-9e13deaef919 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:11.046772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ee974-1b0a-415d-bd3d-2c72c18680f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:11.047648 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ee974-1b0a-415d-bd3d-2c72c18680f8 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:11.048030 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ee974-1b0a-415d-bd3d-2c72c18680f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:11.048432 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ee974-1b0a-415d-bd3d-2c72c18680f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:11.049210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ee974-1b0a-415d-bd3d-2c72c18680f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:11.051141 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ee974-1b0a-415d-bd3d-2c72c18680f8 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:11.063452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1015/2045] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:11.068339 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3834c-7e8e-4b5e-a416-8265881be2e8 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:11.080229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3834c-7e8e-4b5e-a416-8265881be2e8 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:11.085239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc51ff-cb29-4884-924e-53343bcb2d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:11.085932 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc51ff-cb29-4884-924e-53343bcb2d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:11.086330 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc51ff-cb29-4884-924e-53343bcb2d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:11.086489 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc51ff-cb29-4884-924e-53343bcb2d7b admin admin] PATH_INFO: `/v3/projects/dfe2bf21e1214199ba555aa0db27e2fe/users/e733f7878af34f66bdac8f4f5caea026/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:11.110807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc51ff-cb29-4884-924e-53343bcb2d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfe2bf21e1214199ba555aa0db27e2fe, user_id=e733f7878af34f66bdac8f4f5caea026, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:11.115801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc51ff-cb29-4884-924e-53343bcb2d7b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:11.194858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1031/2046] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:11 2022] PUT /identity/v3/projects/dfe2bf21e1214199ba555aa0db27e2fe/users/e733f7878af34f66bdac8f4f5caea026/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:11.206517 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b32df-8b24-4998-aa9a-64b9aeca5c27 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:11.206798 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b32df-8b24-4998-aa9a-64b9aeca5c27 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:11.207125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b32df-8b24-4998-aa9a-64b9aeca5c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:11.236632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-9d5b32df-8b24-4998-aa9a-64b9aeca5c27 None None] MFA Rules not processed for user `e733f7878af34f66bdac8f4f5caea026`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:11.237919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9d5b32df-8b24-4998-aa9a-64b9aeca5c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:11.269774 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:11.305147 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1016/2047] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:11.309958 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc51ff-cb29-4884-924e-53343bcb2d7b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:11.380988 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc51ff-cb29-4884-924e-53343bcb2d7b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:11.386016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038002b-5f75-4d36-ac09-19445b43d570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:11.386591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038002b-5f75-4d36-ac09-19445b43d570 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:11.386878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038002b-5f75-4d36-ac09-19445b43d570 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:11.387178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038002b-5f75-4d36-ac09-19445b43d570 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:11.387815 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038002b-5f75-4d36-ac09-19445b43d570 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:11.389766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038002b-5f75-4d36-ac09-19445b43d570 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:11.489413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1032/2048] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:11 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:11.490616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b32df-8b24-4998-aa9a-64b9aeca5c27 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:11.503996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b32df-8b24-4998-aa9a-64b9aeca5c27 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:11.578716 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e355469-83fc-4085-894c-0eaa3a6567eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:11.579576 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e355469-83fc-4085-894c-0eaa3a6567eb admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:11.579858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e355469-83fc-4085-894c-0eaa3a6567eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:11.580129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e355469-83fc-4085-894c-0eaa3a6567eb admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:11.580791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e355469-83fc-4085-894c-0eaa3a6567eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:11.582802 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e355469-83fc-4085-894c-0eaa3a6567eb admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:11.691713 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1017/2049] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:11 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:11.695539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038002b-5f75-4d36-ac09-19445b43d570 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:11.706173 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038002b-5f75-4d36-ac09-19445b43d570 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:11.710717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d3cbd-b98b-4ddf-9cda-cd5b60d9331b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:11.714800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d3cbd-b98b-4ddf-9cda-cd5b60d9331b admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:11.714800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d3cbd-b98b-4ddf-9cda-cd5b60d9331b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:11.714800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d3cbd-b98b-4ddf-9cda-cd5b60d9331b admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:11.714800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d3cbd-b98b-4ddf-9cda-cd5b60d9331b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:11.714800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d3cbd-b98b-4ddf-9cda-cd5b60d9331b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:11.725260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1033/2050] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:11.729251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e355469-83fc-4085-894c-0eaa3a6567eb admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:11.739783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e355469-83fc-4085-894c-0eaa3a6567eb admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:11.744360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ea326-c91e-4cc5-92c0-48658e69525c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:11.745046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ea326-c91e-4cc5-92c0-48658e69525c admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:11.745330 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ea326-c91e-4cc5-92c0-48658e69525c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:11.745606 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ea326-c91e-4cc5-92c0-48658e69525c admin admin] PATH_INFO: `/v3/projects/6c93873937b44ce2b9e5fd2197d2bf71/users/933f65012ba941b59b84250f74d92916/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:11.760021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ea326-c91e-4cc5-92c0-48658e69525c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c93873937b44ce2b9e5fd2197d2bf71, user_id=933f65012ba941b59b84250f74d92916, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:11.762792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ea326-c91e-4cc5-92c0-48658e69525c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:11.848943 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1018/2051] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:11 2022] PUT /identity/v3/projects/6c93873937b44ce2b9e5fd2197d2bf71/users/933f65012ba941b59b84250f74d92916/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:11.853907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d3cbd-b98b-4ddf-9cda-cd5b60d9331b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:11.873642 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d3cbd-b98b-4ddf-9cda-cd5b60d9331b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:11.927103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:11.942469 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd229301-b75b-4c31-a9b0-27937a47a4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:11.943079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd229301-b75b-4c31-a9b0-27937a47a4df admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:11.943390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd229301-b75b-4c31-a9b0-27937a47a4df admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:11.943650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd229301-b75b-4c31-a9b0-27937a47a4df admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:11.944297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd229301-b75b-4c31-a9b0-27937a47a4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:11.945993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd229301-b75b-4c31-a9b0-27937a47a4df admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:11.957566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1034/2052] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:11.961636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ea326-c91e-4cc5-92c0-48658e69525c admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:11.972845 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ea326-c91e-4cc5-92c0-48658e69525c admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:11.978681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa003b-b752-4717-89ca-61e746878be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:11.979905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa003b-b752-4717-89ca-61e746878be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:11.980369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa003b-b752-4717-89ca-61e746878be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:11.980664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa003b-b752-4717-89ca-61e746878be2 admin admin] PATH_INFO: `/v3/projects/6c93873937b44ce2b9e5fd2197d2bf71/users/933f65012ba941b59b84250f74d92916/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:11.983799 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa003b-b752-4717-89ca-61e746878be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c93873937b44ce2b9e5fd2197d2bf71, user_id=933f65012ba941b59b84250f74d92916, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:11.988750 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa003b-b752-4717-89ca-61e746878be2 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:11.998796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1019/2053] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:11 2022] PUT /identity/v3/projects/6c93873937b44ce2b9e5fd2197d2bf71/users/933f65012ba941b59b84250f74d92916/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:12.008886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a187774-d727-40c5-ab32-5013d8fcac92 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:12.009444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a187774-d727-40c5-ab32-5013d8fcac92 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:12.010081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a187774-d727-40c5-ab32-5013d8fcac92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:12.059611 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-7a187774-d727-40c5-ab32-5013d8fcac92 None None] MFA Rules not processed for user `933f65012ba941b59b84250f74d92916`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:12.060918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7a187774-d727-40c5-ab32-5013d8fcac92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:12.172031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1035/2054] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:12.172237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa003b-b752-4717-89ca-61e746878be2 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:12.172237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-18aa003b-b752-4717-89ca-61e746878be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:12.264978 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa003b-b752-4717-89ca-61e746878be2 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:12.269536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83616b-3559-48a1-8ffa-3432411e9f50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:12.270325 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83616b-3559-48a1-8ffa-3432411e9f50 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:12.270615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83616b-3559-48a1-8ffa-3432411e9f50 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:12.270892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83616b-3559-48a1-8ffa-3432411e9f50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:12.340785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:12.364522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83616b-3559-48a1-8ffa-3432411e9f50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:12.366315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83616b-3559-48a1-8ffa-3432411e9f50 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:12.376361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1020/2055] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:13.370856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3ea8c-fa79-43ae-9ee3-949d2ca7ac7e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:13.371322 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3ea8c-fa79-43ae-9ee3-949d2ca7ac7e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:13.371413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3ea8c-fa79-43ae-9ee3-949d2ca7ac7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:13.394633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83616b-3559-48a1-8ffa-3432411e9f50 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:13.395175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-2af3ea8c-fa79-43ae-9ee3-949d2ca7ac7e None None] MFA Rules not processed for user `83fa9e38ff6d4620aae391266c82fa1c`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:13.396009 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2af3ea8c-fa79-43ae-9ee3-949d2ca7ac7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:13.396219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4d83616b-3559-48a1-8ffa-3432411e9f50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:13.414221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83616b-3559-48a1-8ffa-3432411e9f50 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:13.418757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45143bd2-2ad3-4cf9-bbc1-734fa785676c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:13.419576 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45143bd2-2ad3-4cf9-bbc1-734fa785676c service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:13.420007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45143bd2-2ad3-4cf9-bbc1-734fa785676c service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:13.420587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45143bd2-2ad3-4cf9-bbc1-734fa785676c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:13.423173 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-45143bd2-2ad3-4cf9-bbc1-734fa785676c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:13.473684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1036/2056] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:13.516629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45143bd2-2ad3-4cf9-bbc1-734fa785676c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:13.518486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45143bd2-2ad3-4cf9-bbc1-734fa785676c service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:13.538862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1021/2057] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:53:15.570759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3ea8c-fa79-43ae-9ee3-949d2ca7ac7e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:15.727350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3ea8c-fa79-43ae-9ee3-949d2ca7ac7e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:15.739929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632c4b9-a5ec-4bce-ae55-58b4be1ef19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:15.740410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632c4b9-a5ec-4bce-ae55-58b4be1ef19d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:15.740930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632c4b9-a5ec-4bce-ae55-58b4be1ef19d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:15.741390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632c4b9-a5ec-4bce-ae55-58b4be1ef19d admin admin] PATH_INFO: `/v3/users/e733f7878af34f66bdac8f4f5caea026` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:15.743414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632c4b9-a5ec-4bce-ae55-58b4be1ef19d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e733f7878af34f66bdac8f4f5caea026)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:15.746365 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632c4b9-a5ec-4bce-ae55-58b4be1ef19d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:15.783647 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:16.128259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1632c4b9-a5ec-4bce-ae55-58b4be1ef19d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e733f7878af34f66bdac8f4f5caea026'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:16.138339 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1632c4b9-a5ec-4bce-ae55-58b4be1ef19d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e733f7878af34f66bdac8f4f5caea026'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:16.205894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1632c4b9-a5ec-4bce-ae55-58b4be1ef19d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e733f7878af34f66bdac8f4f5caea026'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:16.206729 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1632c4b9-a5ec-4bce-ae55-58b4be1ef19d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e733f7878af34f66bdac8f4f5caea026'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:16.220210 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1632c4b9-a5ec-4bce-ae55-58b4be1ef19d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e733f7878af34f66bdac8f4f5caea026'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:16.226281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1037/2058] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:15 2022] DELETE /identity/v3/users/e733f7878af34f66bdac8f4f5caea026 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:16.230930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8b0bf-6e52-4832-9f84-a43a6457c24c None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:16.231440 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8b0bf-6e52-4832-9f84-a43a6457c24c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:16.231535 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8b0bf-6e52-4832-9f84-a43a6457c24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:16.251255 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-98b8b0bf-6e52-4832-9f84-a43a6457c24c None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:16.252460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-98b8b0bf-6e52-4832-9f84-a43a6457c24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:16.329600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1022/2059] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:16.341756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632c4b9-a5ec-4bce-ae55-58b4be1ef19d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:16.343857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-1632c4b9-a5ec-4bce-ae55-58b4be1ef19d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:16.404801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632c4b9-a5ec-4bce-ae55-58b4be1ef19d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:16.409422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ba193-a7e6-424a-a6a6-896db68410c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:16.410028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ba193-a7e6-424a-a6a6-896db68410c2 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:16.410315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ba193-a7e6-424a-a6a6-896db68410c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:16.410714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ba193-a7e6-424a-a6a6-896db68410c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:16.423843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ba193-a7e6-424a-a6a6-896db68410c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:16.423843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ba193-a7e6-424a-a6a6-896db68410c2 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:16.434285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1038/2060] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:16.446025 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dae4b-4ee3-42df-b3b0-2c5dabbb63c6 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:16.446249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dae4b-4ee3-42df-b3b0-2c5dabbb63c6 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:16.446526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dae4b-4ee3-42df-b3b0-2c5dabbb63c6 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:16.448812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1023/2061] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 24 07:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:53:16.467073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf0cff-115d-4455-9f05-fcee6079a0e7 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:16.467607 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf0cff-115d-4455-9f05-fcee6079a0e7 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:16.468048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf0cff-115d-4455-9f05-fcee6079a0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:16.497411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-f7bf0cff-115d-4455-9f05-fcee6079a0e7 None None] MFA Rules not processed for user `ea3816d5ae054741ac2c5a2eec3eb617`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:16.499197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f7bf0cff-115d-4455-9f05-fcee6079a0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:16.595224 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1039/2062] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 24 07:53:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:16.614959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dae4b-4ee3-42df-b3b0-2c5dabbb63c6 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:16.616943 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fd2dae4b-4ee3-42df-b3b0-2c5dabbb63c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:16.643414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf0cff-115d-4455-9f05-fcee6079a0e7 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:16.645078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f7bf0cff-115d-4455-9f05-fcee6079a0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:16.646994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dae4b-4ee3-42df-b3b0-2c5dabbb63c6 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:16.653391 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c8fe7-ebfc-4ced-b06a-158dd59992be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:16.659860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c8fe7-ebfc-4ced-b06a-158dd59992be admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:16.659860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c8fe7-ebfc-4ced-b06a-158dd59992be admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:16.659860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c8fe7-ebfc-4ced-b06a-158dd59992be admin admin] PATH_INFO: `/v3/users/a610194aaf534d3aabb56c9443b5d56d` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:16.659860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c8fe7-ebfc-4ced-b06a-158dd59992be admin admin] RBAC: Authorizing `identity:delete_user(user_id=a610194aaf534d3aabb56c9443b5d56d)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:16.660616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c8fe7-ebfc-4ced-b06a-158dd59992be admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:16.661795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf0cff-115d-4455-9f05-fcee6079a0e7 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:16.680831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f6b97-4329-4e15-a021-650cd2ce663f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:16.681483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f6b97-4329-4e15-a021-650cd2ce663f service placement] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:16.687383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f6b97-4329-4e15-a021-650cd2ce663f service placement] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:16.687383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f6b97-4329-4e15-a021-650cd2ce663f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:16.707670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f6b97-4329-4e15-a021-650cd2ce663f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:16.710601 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f6b97-4329-4e15-a021-650cd2ce663f service placement] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:16.731032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1040/2063] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:53:16.920430 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-954c8fe7-ebfc-4ced-b06a-158dd59992be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a610194aaf534d3aabb56c9443b5d56d'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:16.946063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f6b97-4329-4e15-a021-650cd2ce663f service placement] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:16.947742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a25f6b97-4329-4e15-a021-650cd2ce663f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:16.965828 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f6b97-4329-4e15-a021-650cd2ce663f service placement] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:16.972033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:16.973350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:16.973943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:16.974342 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] PATH_INFO: `/v3/projects/dfe2bf21e1214199ba555aa0db27e2fe` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:16.976044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfe2bf21e1214199ba555aa0db27e2fe)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:16.978241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:17.002779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:17.010632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-954c8fe7-ebfc-4ced-b06a-158dd59992be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a610194aaf534d3aabb56c9443b5d56d'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.011297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-954c8fe7-ebfc-4ced-b06a-158dd59992be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a610194aaf534d3aabb56c9443b5d56d'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.019772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-954c8fe7-ebfc-4ced-b06a-158dd59992be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a610194aaf534d3aabb56c9443b5d56d'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.028028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:17.037649 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-954c8fe7-ebfc-4ced-b06a-158dd59992be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a610194aaf534d3aabb56c9443b5d56d'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.054327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1024/2064] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:16 2022] DELETE /identity/v3/users/a610194aaf534d3aabb56c9443b5d56d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:17.115453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfe2bf21e1214199ba555aa0db27e2fe', 'request_id': 'req-a00614d0-b99b-47ee-88d2-b62677d529ac'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.121760 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfe2bf21e1214199ba555aa0db27e2fe', 'request_id': 'req-a00614d0-b99b-47ee-88d2-b62677d529ac'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.127955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfe2bf21e1214199ba555aa0db27e2fe', 'request_id': 'req-a00614d0-b99b-47ee-88d2-b62677d529ac'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.199240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] The token cache is being invalidate because project dfe2bf21e1214199ba555aa0db27e2fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:17.199531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.200369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.202157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1041/2065] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:16 2022] DELETE /identity/v3/projects/dfe2bf21e1214199ba555aa0db27e2fe => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:17.208504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c8fe7-ebfc-4ced-b06a-158dd59992be admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:17.213872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-954c8fe7-ebfc-4ced-b06a-158dd59992be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:17.296204 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c8fe7-ebfc-4ced-b06a-158dd59992be admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:17.370816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487bafe0-22fb-4d77-967e-1f83e9b0b522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:17.371453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bafe0-22fb-4d77-967e-1f83e9b0b522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:17.377349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bafe0-22fb-4d77-967e-1f83e9b0b522 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:17.377349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bafe0-22fb-4d77-967e-1f83e9b0b522 admin admin] PATH_INFO: `/v3/users/933f65012ba941b59b84250f74d92916` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:17.377349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487bafe0-22fb-4d77-967e-1f83e9b0b522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=933f65012ba941b59b84250f74d92916)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:17.377349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487bafe0-22fb-4d77-967e-1f83e9b0b522 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:17.491667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:17.493548 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:17.507329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00614d0-b99b-47ee-88d2-b62677d529ac admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:17.512253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:17.512914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:17.513196 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:17.513467 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5 admin admin] PATH_INFO: `/v3/projects/af7797a0a97b4debb3492e864b6ca1b1` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:17.514598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af7797a0a97b4debb3492e864b6ca1b1)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:17.516610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:17.569796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-487bafe0-22fb-4d77-967e-1f83e9b0b522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '933f65012ba941b59b84250f74d92916'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.685067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-487bafe0-22fb-4d77-967e-1f83e9b0b522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '933f65012ba941b59b84250f74d92916'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.685736 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-487bafe0-22fb-4d77-967e-1f83e9b0b522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '933f65012ba941b59b84250f74d92916'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.696080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-487bafe0-22fb-4d77-967e-1f83e9b0b522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '933f65012ba941b59b84250f74d92916'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.711731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af7797a0a97b4debb3492e864b6ca1b1', 'request_id': 'req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.713807 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-487bafe0-22fb-4d77-967e-1f83e9b0b522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '933f65012ba941b59b84250f74d92916'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.720968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af7797a0a97b4debb3492e864b6ca1b1', 'request_id': 'req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.728991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af7797a0a97b4debb3492e864b6ca1b1', 'request_id': 'req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.729416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1025/2066] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:17 2022] DELETE /identity/v3/users/933f65012ba941b59b84250f74d92916 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:17.836191 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5 admin admin] The token cache is being invalidate because project af7797a0a97b4debb3492e864b6ca1b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:17.836493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.837179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e8c4cc48-f839-4e48-b39d-4cbf4b0a09c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:17.838733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1042/2067] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:17 2022] DELETE /identity/v3/projects/af7797a0a97b4debb3492e864b6ca1b1 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:17.843834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1011ec-f563-46a5-8937-768c6699873d None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:17.844127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1011ec-f563-46a5-8937-768c6699873d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:17.844405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1011ec-f563-46a5-8937-768c6699873d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:17.864022 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-0c1011ec-f563-46a5-8937-768c6699873d None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:17.865258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0c1011ec-f563-46a5-8937-768c6699873d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:17.870420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:17.929852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1026/2068] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:17.938343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ff70a-cb67-4b7e-a3a9-05c8fe99817a None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:17.938876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ff70a-cb67-4b7e-a3a9-05c8fe99817a None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:17.939165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ff70a-cb67-4b7e-a3a9-05c8fe99817a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:17.973954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-4b4ff70a-cb67-4b7e-a3a9-05c8fe99817a None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:17.973954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4b4ff70a-cb67-4b7e-a3a9-05c8fe99817a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:17.983539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1043/2069] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:17.988935 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1011ec-f563-46a5-8937-768c6699873d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:18.001865 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1011ec-f563-46a5-8937-768c6699873d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:18.007513 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a4f71-e7f6-4395-a17b-6d424651339f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:18.007513 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a4f71-e7f6-4395-a17b-6d424651339f admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:18.007789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a4f71-e7f6-4395-a17b-6d424651339f admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:18.008111 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a4f71-e7f6-4395-a17b-6d424651339f admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:18.008793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a4f71-e7f6-4395-a17b-6d424651339f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:53:18.009098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a4f71-e7f6-4395-a17b-6d424651339f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:18.010716 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a4f71-e7f6-4395-a17b-6d424651339f admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:18.027442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1027/2070] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:53:18.029794 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ff70a-cb67-4b7e-a3a9-05c8fe99817a None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:18.043794 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ff70a-cb67-4b7e-a3a9-05c8fe99817a None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:18.049933 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b97550-d653-45dc-99fd-6d02d2407482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:18.050597 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b97550-d653-45dc-99fd-6d02d2407482 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:18.050879 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b97550-d653-45dc-99fd-6d02d2407482 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:18.051145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b97550-d653-45dc-99fd-6d02d2407482 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:18.051769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b97550-d653-45dc-99fd-6d02d2407482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:18.054010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b97550-d653-45dc-99fd-6d02d2407482 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:18.151569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1044/2071] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:18 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:18.156569 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a4f71-e7f6-4395-a17b-6d424651339f admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:18.167108 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a4f71-e7f6-4395-a17b-6d424651339f admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:18.221168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b97550-d653-45dc-99fd-6d02d2407482 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:18.222846 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-73b97550-d653-45dc-99fd-6d02d2407482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:18.248759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a63321-0145-4d97-aadc-a6e7076fad31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:18.249603 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a63321-0145-4d97-aadc-a6e7076fad31 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:18.250066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a63321-0145-4d97-aadc-a6e7076fad31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:18.250360 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a63321-0145-4d97-aadc-a6e7076fad31 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:18.250997 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a63321-0145-4d97-aadc-a6e7076fad31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:18.253598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a63321-0145-4d97-aadc-a6e7076fad31 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:18.318074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b97550-d653-45dc-99fd-6d02d2407482 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:18.323010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:18.323644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:18.323958 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:18.324196 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c admin admin] PATH_INFO: `/v3/projects/6c93873937b44ce2b9e5fd2197d2bf71` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:18.325515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c93873937b44ce2b9e5fd2197d2bf71)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:18.327414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:18.329880 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:18.365068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1028/2072] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:18 2022] POST /identity/v3/users => generated 283 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:18.368926 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a63321-0145-4d97-aadc-a6e7076fad31 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:18.379228 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a63321-0145-4d97-aadc-a6e7076fad31 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:18.384021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87119bfd-19b3-4dd8-8561-0c75751d9957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:18.384622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87119bfd-19b3-4dd8-8561-0c75751d9957 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:18.384959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87119bfd-19b3-4dd8-8561-0c75751d9957 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:18.385238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87119bfd-19b3-4dd8-8561-0c75751d9957 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:18.386624 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87119bfd-19b3-4dd8-8561-0c75751d9957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:18.388047 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87119bfd-19b3-4dd8-8561-0c75751d9957 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:18.401132 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1029/2073] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:53:18.404962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87119bfd-19b3-4dd8-8561-0c75751d9957 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:18.417500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87119bfd-19b3-4dd8-8561-0c75751d9957 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:18.427684 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd1129-16f5-4a38-b5a5-3d808695d80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:18.428381 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd1129-16f5-4a38-b5a5-3d808695d80d admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:18.428561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd1129-16f5-4a38-b5a5-3d808695d80d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:18.429265 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd1129-16f5-4a38-b5a5-3d808695d80d admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:18.429497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd1129-16f5-4a38-b5a5-3d808695d80d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:18.431169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd1129-16f5-4a38-b5a5-3d808695d80d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:18.445482 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1030/2074] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:18.449705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd1129-16f5-4a38-b5a5-3d808695d80d admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:18.466625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd1129-16f5-4a38-b5a5-3d808695d80d admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:18.481304 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c93873937b44ce2b9e5fd2197d2bf71', 'request_id': 'req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:18.499315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c93873937b44ce2b9e5fd2197d2bf71', 'request_id': 'req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:18.510134 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c93873937b44ce2b9e5fd2197d2bf71', 'request_id': 'req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:18.566844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99485ac-5704-4062-9360-2c7fd9a912c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:18.567505 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99485ac-5704-4062-9360-2c7fd9a912c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:18.567785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99485ac-5704-4062-9360-2c7fd9a912c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:18.568063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99485ac-5704-4062-9360-2c7fd9a912c7 admin admin] PATH_INFO: `/v3/projects/6c92d3a55c5f4db69e29c2043e2e0765/users/f3735a60edb44f459e9fdb91aeef47fb/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:18.580261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:18.590147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c admin admin] The token cache is being invalidate because project 6c93873937b44ce2b9e5fd2197d2bf71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:18.590497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:18.591303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99485ac-5704-4062-9360-2c7fd9a912c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c92d3a55c5f4db69e29c2043e2e0765, user_id=f3735a60edb44f459e9fdb91aeef47fb, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:18.591422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-21aa5327-a334-4cfd-a8d4-3a11a6f4d59c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:18.593318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1045/2075] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:18 2022] DELETE /identity/v3/projects/6c93873937b44ce2b9e5fd2197d2bf71 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:18.604261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99485ac-5704-4062-9360-2c7fd9a912c7 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:18.704742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1031/2076] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:18 2022] PUT /identity/v3/projects/6c92d3a55c5f4db69e29c2043e2e0765/users/f3735a60edb44f459e9fdb91aeef47fb/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:18.718425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b40b8f-5706-4b2a-920a-2d709858192c None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:18.718938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b40b8f-5706-4b2a-920a-2d709858192c None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:18.718938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b40b8f-5706-4b2a-920a-2d709858192c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:18.774734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-57b40b8f-5706-4b2a-920a-2d709858192c None None] MFA Rules not processed for user `f3735a60edb44f459e9fdb91aeef47fb`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:18.776459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-57b40b8f-5706-4b2a-920a-2d709858192c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:18.861261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1046/2077] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:18.866178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99485ac-5704-4062-9360-2c7fd9a912c7 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:18.867921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f99485ac-5704-4062-9360-2c7fd9a912c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:18.944755 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99485ac-5704-4062-9360-2c7fd9a912c7 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:18.949533 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced08e0e-8425-4c10-a8b1-987a60c146fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:18.950137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced08e0e-8425-4c10-a8b1-987a60c146fd admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:18.950419 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced08e0e-8425-4c10-a8b1-987a60c146fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:18.950689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced08e0e-8425-4c10-a8b1-987a60c146fd admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:18.951311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced08e0e-8425-4c10-a8b1-987a60c146fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:18.953288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced08e0e-8425-4c10-a8b1-987a60c146fd admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:19.078389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1032/2078] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:18 2022] POST /identity/v3/projects => generated 365 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:19.082856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b40b8f-5706-4b2a-920a-2d709858192c None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:19.095330 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b40b8f-5706-4b2a-920a-2d709858192c None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:19.154306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7b8ff-1873-4d92-83f8-4d5322ad37c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:19.155033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7b8ff-1873-4d92-83f8-4d5322ad37c7 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:19.155315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7b8ff-1873-4d92-83f8-4d5322ad37c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:19.155584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7b8ff-1873-4d92-83f8-4d5322ad37c7 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:19.157055 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7b8ff-1873-4d92-83f8-4d5322ad37c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:19.158970 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7b8ff-1873-4d92-83f8-4d5322ad37c7 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:19.269612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1047/2079] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:19 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:19.273738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced08e0e-8425-4c10-a8b1-987a60c146fd admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:19.285747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced08e0e-8425-4c10-a8b1-987a60c146fd admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:19.291282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35774090-8a7a-409f-aabe-75d2f817a8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:19.292011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35774090-8a7a-409f-aabe-75d2f817a8ed admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:19.292696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35774090-8a7a-409f-aabe-75d2f817a8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:19.292696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35774090-8a7a-409f-aabe-75d2f817a8ed admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:19.293355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35774090-8a7a-409f-aabe-75d2f817a8ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:19.295076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35774090-8a7a-409f-aabe-75d2f817a8ed admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:19.312507 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1033/2080] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:53:19.313765 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7b8ff-1873-4d92-83f8-4d5322ad37c7 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:19.324683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7b8ff-1873-4d92-83f8-4d5322ad37c7 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:19.331498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3841f-6bbd-421e-a5ee-3f373c7ed5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:19.332400 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3841f-6bbd-421e-a5ee-3f373c7ed5e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:19.332703 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3841f-6bbd-421e-a5ee-3f373c7ed5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:19.332985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3841f-6bbd-421e-a5ee-3f373c7ed5e8 admin admin] PATH_INFO: `/v3/projects/f1b1b3574cd84b1c80511df952b62f38/users/c01fba8d0aa8409097e5f6dec6c0ccc3/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:19.336912 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:19.350437 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3841f-6bbd-421e-a5ee-3f373c7ed5e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b1b3574cd84b1c80511df952b62f38, user_id=c01fba8d0aa8409097e5f6dec6c0ccc3, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:19.353488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3841f-6bbd-421e-a5ee-3f373c7ed5e8 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:19.442613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1048/2081] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:19 2022] PUT /identity/v3/projects/f1b1b3574cd84b1c80511df952b62f38/users/c01fba8d0aa8409097e5f6dec6c0ccc3/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:19.446824 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35774090-8a7a-409f-aabe-75d2f817a8ed admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:19.458067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35774090-8a7a-409f-aabe-75d2f817a8ed admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:19.516898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d4630-7050-482a-9908-efebfaaaca1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:19.517556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d4630-7050-482a-9908-efebfaaaca1a admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:19.517973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d4630-7050-482a-9908-efebfaaaca1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:19.518258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d4630-7050-482a-9908-efebfaaaca1a admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:19.518908 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d4630-7050-482a-9908-efebfaaaca1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:19.521141 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d4630-7050-482a-9908-efebfaaaca1a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:19.532694 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1034/2082] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:19.537218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3841f-6bbd-421e-a5ee-3f373c7ed5e8 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:19.548249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3841f-6bbd-421e-a5ee-3f373c7ed5e8 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:19.553840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592bf4c-01d5-4780-b7e5-c581aebbf5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:19.553954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592bf4c-01d5-4780-b7e5-c581aebbf5e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:19.554331 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592bf4c-01d5-4780-b7e5-c581aebbf5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:19.554636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592bf4c-01d5-4780-b7e5-c581aebbf5e9 admin admin] PATH_INFO: `/v3/projects/f1b1b3574cd84b1c80511df952b62f38/users/c01fba8d0aa8409097e5f6dec6c0ccc3/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:19.556586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592bf4c-01d5-4780-b7e5-c581aebbf5e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b1b3574cd84b1c80511df952b62f38, user_id=c01fba8d0aa8409097e5f6dec6c0ccc3, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:19.559524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592bf4c-01d5-4780-b7e5-c581aebbf5e9 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:19.570819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1049/2083] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:19 2022] PUT /identity/v3/projects/f1b1b3574cd84b1c80511df952b62f38/users/c01fba8d0aa8409097e5f6dec6c0ccc3/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:19.583324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0a903-541a-4d72-a6d9-e897d2b5d5f9 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:19.583619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0a903-541a-4d72-a6d9-e897d2b5d5f9 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:19.584123 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0a903-541a-4d72-a6d9-e897d2b5d5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:19.614628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-dad0a903-541a-4d72-a6d9-e897d2b5d5f9 None None] MFA Rules not processed for user `c01fba8d0aa8409097e5f6dec6c0ccc3`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:19.615930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-dad0a903-541a-4d72-a6d9-e897d2b5d5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:19.664782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:19.687668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1035/2084] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:19.699362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592bf4c-01d5-4780-b7e5-c581aebbf5e9 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:19.701142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8592bf4c-01d5-4780-b7e5-c581aebbf5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:19.761228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592bf4c-01d5-4780-b7e5-c581aebbf5e9 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:19.766417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1c522-185a-41d7-aafd-65d884750329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:19.767181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1c522-185a-41d7-aafd-65d884750329 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:19.767463 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1c522-185a-41d7-aafd-65d884750329 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:19.767739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1c522-185a-41d7-aafd-65d884750329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:19.777242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1c522-185a-41d7-aafd-65d884750329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:19.778598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1c522-185a-41d7-aafd-65d884750329 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:19.789317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1050/2085] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:19.860962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0a903-541a-4d72-a6d9-e897d2b5d5f9 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:19.862390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-dad0a903-541a-4d72-a6d9-e897d2b5d5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:19.931638 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0a903-541a-4d72-a6d9-e897d2b5d5f9 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:19.936684 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22018ca-18dc-4a84-b510-ae319cac9117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:19.937922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22018ca-18dc-4a84-b510-ae319cac9117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:19.938201 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22018ca-18dc-4a84-b510-ae319cac9117 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:19.938477 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22018ca-18dc-4a84-b510-ae319cac9117 admin admin] PATH_INFO: `/v3/users/97c18d4518fa4e3abaac5bd9bc1fd02e` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:19.939768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22018ca-18dc-4a84-b510-ae319cac9117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c18d4518fa4e3abaac5bd9bc1fd02e)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:19.941828 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22018ca-18dc-4a84-b510-ae319cac9117 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:20.058752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1c522-185a-41d7-aafd-65d884750329 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:20.068131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1c522-185a-41d7-aafd-65d884750329 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:20.071862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28ea97-ae05-4f5b-8370-52b166165719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:20.072461 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28ea97-ae05-4f5b-8370-52b166165719 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:20.072788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28ea97-ae05-4f5b-8370-52b166165719 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:20.073065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28ea97-ae05-4f5b-8370-52b166165719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:20.125253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d22018ca-18dc-4a84-b510-ae319cac9117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c18d4518fa4e3abaac5bd9bc1fd02e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:20.137152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28ea97-ae05-4f5b-8370-52b166165719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:20.138628 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28ea97-ae05-4f5b-8370-52b166165719 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:20.148519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1051/2086] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:20.244509 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d22018ca-18dc-4a84-b510-ae319cac9117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c18d4518fa4e3abaac5bd9bc1fd02e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:20.245226 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d22018ca-18dc-4a84-b510-ae319cac9117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c18d4518fa4e3abaac5bd9bc1fd02e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:20.252949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d22018ca-18dc-4a84-b510-ae319cac9117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c18d4518fa4e3abaac5bd9bc1fd02e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:20.267047 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d22018ca-18dc-4a84-b510-ae319cac9117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c18d4518fa4e3abaac5bd9bc1fd02e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:20.282258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1036/2087] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:19 2022] DELETE /identity/v3/users/97c18d4518fa4e3abaac5bd9bc1fd02e => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:20.285620 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323751c2-636b-4f06-8d5b-ef4c7b64befa None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:20.285936 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323751c2-636b-4f06-8d5b-ef4c7b64befa None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:20.286230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323751c2-636b-4f06-8d5b-ef4c7b64befa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:20.305590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-323751c2-636b-4f06-8d5b-ef4c7b64befa None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:20.306759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-323751c2-636b-4f06-8d5b-ef4c7b64befa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:20.365921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1052/2088] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:20.375840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22018ca-18dc-4a84-b510-ae319cac9117 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:20.377251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d22018ca-18dc-4a84-b510-ae319cac9117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:20.440749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22018ca-18dc-4a84-b510-ae319cac9117 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:20.444521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da33034-b202-4085-bab3-1f8ccfdd48fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:20.445152 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da33034-b202-4085-bab3-1f8ccfdd48fc service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:20.445440 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da33034-b202-4085-bab3-1f8ccfdd48fc service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:20.445731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da33034-b202-4085-bab3-1f8ccfdd48fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:20.455199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da33034-b202-4085-bab3-1f8ccfdd48fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:20.456681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da33034-b202-4085-bab3-1f8ccfdd48fc service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:20.466149 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1037/2089] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:20.985027 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323751c2-636b-4f06-8d5b-ef4c7b64befa None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:20.987669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-323751c2-636b-4f06-8d5b-ef4c7b64befa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:21.001291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323751c2-636b-4f06-8d5b-ef4c7b64befa None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:21.006061 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:21.006690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:21.007515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:21.007515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] PATH_INFO: `/v3/projects/5ab57f71ced046a3bc9fd451d23e9af8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:21.008358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ab57f71ced046a3bc9fd451d23e9af8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:21.010329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:21.020797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:21.142928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ab57f71ced046a3bc9fd451d23e9af8', 'request_id': 'req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:21.148992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ab57f71ced046a3bc9fd451d23e9af8', 'request_id': 'req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:21.154662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ab57f71ced046a3bc9fd451d23e9af8', 'request_id': 'req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:21.244118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] The token cache is being invalidate because project 5ab57f71ced046a3bc9fd451d23e9af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:21.244361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:21.245155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:21.246602 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1053/2090] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:20 2022] DELETE /identity/v3/projects/5ab57f71ced046a3bc9fd451d23e9af8 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:21.250962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da33034-b202-4085-bab3-1f8ccfdd48fc service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:21.260851 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da33034-b202-4085-bab3-1f8ccfdd48fc service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:21.264356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:21.322759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09da8ed-8533-49ca-968e-2c3e39672b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:21.323420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09da8ed-8533-49ca-968e-2c3e39672b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:21.324005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09da8ed-8533-49ca-968e-2c3e39672b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:21.324517 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09da8ed-8533-49ca-968e-2c3e39672b62 admin admin] PATH_INFO: `/v3/users/a3c4f2d854d24e2489cf774087d3d313` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:21.326082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09da8ed-8533-49ca-968e-2c3e39672b62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3c4f2d854d24e2489cf774087d3d313)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:21.328558 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09da8ed-8533-49ca-968e-2c3e39672b62 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:21.526999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e09da8ed-8533-49ca-968e-2c3e39672b62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3c4f2d854d24e2489cf774087d3d313'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:21.615916 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e09da8ed-8533-49ca-968e-2c3e39672b62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3c4f2d854d24e2489cf774087d3d313'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:21.616543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e09da8ed-8533-49ca-968e-2c3e39672b62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3c4f2d854d24e2489cf774087d3d313'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:21.624237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e09da8ed-8533-49ca-968e-2c3e39672b62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3c4f2d854d24e2489cf774087d3d313'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:21.636729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e09da8ed-8533-49ca-968e-2c3e39672b62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3c4f2d854d24e2489cf774087d3d313'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:21.643698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1038/2091] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:21 2022] DELETE /identity/v3/users/a3c4f2d854d24e2489cf774087d3d313 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:22.138690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:22.140740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:22.168212 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f112216-bbb2-4322-bbd4-9b9a3a5d5e07 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:22.227702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ec864-9e31-4462-ac72-8e7b9bac6713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:22.228362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ec864-9e31-4462-ac72-8e7b9bac6713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:22.228661 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ec864-9e31-4462-ac72-8e7b9bac6713 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:22.228973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ec864-9e31-4462-ac72-8e7b9bac6713 admin admin] PATH_INFO: `/v3/projects/94842f2f01954064991f6bbeb381b94f` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:22.230091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ec864-9e31-4462-ac72-8e7b9bac6713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94842f2f01954064991f6bbeb381b94f)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:22.232050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ec864-9e31-4462-ac72-8e7b9bac6713 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:22.333858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-349ec864-9e31-4462-ac72-8e7b9bac6713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94842f2f01954064991f6bbeb381b94f', 'request_id': 'req-349ec864-9e31-4462-ac72-8e7b9bac6713'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:22.340132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-349ec864-9e31-4462-ac72-8e7b9bac6713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94842f2f01954064991f6bbeb381b94f', 'request_id': 'req-349ec864-9e31-4462-ac72-8e7b9bac6713'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:22.346394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-349ec864-9e31-4462-ac72-8e7b9bac6713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94842f2f01954064991f6bbeb381b94f', 'request_id': 'req-349ec864-9e31-4462-ac72-8e7b9bac6713'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:22.444497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-349ec864-9e31-4462-ac72-8e7b9bac6713 admin admin] The token cache is being invalidate because project 94842f2f01954064991f6bbeb381b94f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:22.444852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-349ec864-9e31-4462-ac72-8e7b9bac6713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:22.445461 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-349ec864-9e31-4462-ac72-8e7b9bac6713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:22.446967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1054/2092] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:22 2022] DELETE /identity/v3/projects/94842f2f01954064991f6bbeb381b94f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:22.452758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb00d1f-4f08-4563-ba15-a0facd662e19 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:22.453071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb00d1f-4f08-4563-ba15-a0facd662e19 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:22.453340 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb00d1f-4f08-4563-ba15-a0facd662e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:22.474893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-4fb00d1f-4f08-4563-ba15-a0facd662e19 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:22.476101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4fb00d1f-4f08-4563-ba15-a0facd662e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:22.558946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1039/2093] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:22.568069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c839c-8fcf-418b-8670-a606addc0fb5 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:22.568398 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c839c-8fcf-418b-8670-a606addc0fb5 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:22.568729 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c839c-8fcf-418b-8670-a606addc0fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:22.589678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-520c839c-8fcf-418b-8670-a606addc0fb5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:22.590926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-520c839c-8fcf-418b-8670-a606addc0fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:22.600050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1055/2094] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:22.604079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb00d1f-4f08-4563-ba15-a0facd662e19 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:22.614866 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb00d1f-4f08-4563-ba15-a0facd662e19 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:22.619816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e52df-ee2b-49c9-a565-c7e1f1b48110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:22.620407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e52df-ee2b-49c9-a565-c7e1f1b48110 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:22.620729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e52df-ee2b-49c9-a565-c7e1f1b48110 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:22.621010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e52df-ee2b-49c9-a565-c7e1f1b48110 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:22.621687 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e52df-ee2b-49c9-a565-c7e1f1b48110 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:53:22.621992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e52df-ee2b-49c9-a565-c7e1f1b48110 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:22.623675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e52df-ee2b-49c9-a565-c7e1f1b48110 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:22.639702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1040/2095] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:53:22.640899 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c839c-8fcf-418b-8670-a606addc0fb5 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:22.650910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c839c-8fcf-418b-8670-a606addc0fb5 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:22.655566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893cc6e-3e0c-4164-9fd5-db54c2cd8f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:22.656201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893cc6e-3e0c-4164-9fd5-db54c2cd8f4b admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:22.656479 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893cc6e-3e0c-4164-9fd5-db54c2cd8f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:22.656781 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893cc6e-3e0c-4164-9fd5-db54c2cd8f4b admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:22.657977 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893cc6e-3e0c-4164-9fd5-db54c2cd8f4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:22.660016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893cc6e-3e0c-4164-9fd5-db54c2cd8f4b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:22.759424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:22.767455 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1056/2096] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:22 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:22.771999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e52df-ee2b-49c9-a565-c7e1f1b48110 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:22.781933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e52df-ee2b-49c9-a565-c7e1f1b48110 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:22.791383 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:22.838612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2962a-2578-4224-8c2b-008e2268203a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:22.839217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2962a-2578-4224-8c2b-008e2268203a admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:22.839499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2962a-2578-4224-8c2b-008e2268203a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:22.839776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2962a-2578-4224-8c2b-008e2268203a admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:22.840415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2962a-2578-4224-8c2b-008e2268203a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:22.842365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2962a-2578-4224-8c2b-008e2268203a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:22.912046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1041/2097] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:22 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:22.913356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893cc6e-3e0c-4164-9fd5-db54c2cd8f4b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:22.923823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893cc6e-3e0c-4164-9fd5-db54c2cd8f4b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:22.928729 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0a925-0d17-43de-9374-b2f3f7453de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:22.929348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0a925-0d17-43de-9374-b2f3f7453de3 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:22.929606 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0a925-0d17-43de-9374-b2f3f7453de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:22.929858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0a925-0d17-43de-9374-b2f3f7453de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:22.930503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0a925-0d17-43de-9374-b2f3f7453de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:22.932097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0a925-0d17-43de-9374-b2f3f7453de3 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:22.942998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1057/2098] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:22.947291 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2962a-2578-4224-8c2b-008e2268203a admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:22.958271 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2962a-2578-4224-8c2b-008e2268203a admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:22.963048 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f0a72-24f8-4d4e-ba0d-1363afdb6771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:22.963651 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f0a72-24f8-4d4e-ba0d-1363afdb6771 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:22.963954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f0a72-24f8-4d4e-ba0d-1363afdb6771 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:22.964208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f0a72-24f8-4d4e-ba0d-1363afdb6771 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:22.964962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f0a72-24f8-4d4e-ba0d-1363afdb6771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:22.966559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f0a72-24f8-4d4e-ba0d-1363afdb6771 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:22.977673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1042/2099] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:22.982092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0a925-0d17-43de-9374-b2f3f7453de3 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:22.992782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0a925-0d17-43de-9374-b2f3f7453de3 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:22.997591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b1fac-5b79-44d5-a3ed-a9efe3172096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:22.998314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b1fac-5b79-44d5-a3ed-a9efe3172096 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:22.998668 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b1fac-5b79-44d5-a3ed-a9efe3172096 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:22.999014 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b1fac-5b79-44d5-a3ed-a9efe3172096 admin admin] PATH_INFO: `/v3/projects/5cf5fee9c60d4cefb9d2ef03b313fb4a/users/2f8e4c1f62874356a67c17ae5eea93f3/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:23.015412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b1fac-5b79-44d5-a3ed-a9efe3172096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cf5fee9c60d4cefb9d2ef03b313fb4a, user_id=2f8e4c1f62874356a67c17ae5eea93f3, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:23.019017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b1fac-5b79-44d5-a3ed-a9efe3172096 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:23.073352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1058/2100] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:22 2022] PUT /identity/v3/projects/5cf5fee9c60d4cefb9d2ef03b313fb4a/users/2f8e4c1f62874356a67c17ae5eea93f3/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:23.087129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334d76d-fb20-41a3-8d9e-4a808de78851 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:23.087443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334d76d-fb20-41a3-8d9e-4a808de78851 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:23.087721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334d76d-fb20-41a3-8d9e-4a808de78851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:23.115135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-6334d76d-fb20-41a3-8d9e-4a808de78851 None None] MFA Rules not processed for user `2f8e4c1f62874356a67c17ae5eea93f3`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:23.116340 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6334d76d-fb20-41a3-8d9e-4a808de78851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:23.180224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1043/2101] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:23.192451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b1fac-5b79-44d5-a3ed-a9efe3172096 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:23.194357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e91b1fac-5b79-44d5-a3ed-a9efe3172096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:23.249466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b1fac-5b79-44d5-a3ed-a9efe3172096 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:23.253662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10adfe40-f35d-4f6d-8fe2-0bed1ab059d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:23.254286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adfe40-f35d-4f6d-8fe2-0bed1ab059d5 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:23.254565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adfe40-f35d-4f6d-8fe2-0bed1ab059d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:23.254836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adfe40-f35d-4f6d-8fe2-0bed1ab059d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:23.263021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10adfe40-f35d-4f6d-8fe2-0bed1ab059d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:23.264439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10adfe40-f35d-4f6d-8fe2-0bed1ab059d5 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:23.273133 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1059/2102] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:23.292263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334d76d-fb20-41a3-8d9e-4a808de78851 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:23.355493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334d76d-fb20-41a3-8d9e-4a808de78851 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:23.360107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688404d-0d49-4806-8be0-dd584eb5dd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:23.361285 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688404d-0d49-4806-8be0-dd584eb5dd27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:23.361568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688404d-0d49-4806-8be0-dd584eb5dd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:23.361842 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688404d-0d49-4806-8be0-dd584eb5dd27 admin admin] PATH_INFO: `/v3/users/2f8e4c1f62874356a67c17ae5eea93f3` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:23.362961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688404d-0d49-4806-8be0-dd584eb5dd27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f8e4c1f62874356a67c17ae5eea93f3)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:23.364960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688404d-0d49-4806-8be0-dd584eb5dd27 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:23.388137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:23.545184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8688404d-0d49-4806-8be0-dd584eb5dd27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f8e4c1f62874356a67c17ae5eea93f3'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:23.638512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8688404d-0d49-4806-8be0-dd584eb5dd27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f8e4c1f62874356a67c17ae5eea93f3'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:23.639271 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8688404d-0d49-4806-8be0-dd584eb5dd27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f8e4c1f62874356a67c17ae5eea93f3'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:23.647492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8688404d-0d49-4806-8be0-dd584eb5dd27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f8e4c1f62874356a67c17ae5eea93f3'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:23.660527 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8688404d-0d49-4806-8be0-dd584eb5dd27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f8e4c1f62874356a67c17ae5eea93f3'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:23.666796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1044/2103] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:23 2022] DELETE /identity/v3/users/2f8e4c1f62874356a67c17ae5eea93f3 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:23.671352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81206e91-b48a-4c8a-8c78-9f4c8f259411 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:23.671649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81206e91-b48a-4c8a-8c78-9f4c8f259411 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:23.671938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81206e91-b48a-4c8a-8c78-9f4c8f259411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:23.696560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-81206e91-b48a-4c8a-8c78-9f4c8f259411 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:23.697973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-81206e91-b48a-4c8a-8c78-9f4c8f259411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:23.768144 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1060/2104] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:23.777954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688404d-0d49-4806-8be0-dd584eb5dd27 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:23.779646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-8688404d-0d49-4806-8be0-dd584eb5dd27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:23.837740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81206e91-b48a-4c8a-8c78-9f4c8f259411 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:23.839138 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-81206e91-b48a-4c8a-8c78-9f4c8f259411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:23.840602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688404d-0d49-4806-8be0-dd584eb5dd27 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:23.844275 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224d627-7378-4d65-b607-50e9ed036806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:23.845520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d627-7378-4d65-b607-50e9ed036806 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:23.845911 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d627-7378-4d65-b607-50e9ed036806 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:23.846192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d627-7378-4d65-b607-50e9ed036806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:23.852328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81206e91-b48a-4c8a-8c78-9f4c8f259411 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:23.855706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224d627-7378-4d65-b607-50e9ed036806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:23.857106 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4515527-1aeb-43f9-a709-18f6fb507827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:23.857235 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224d627-7378-4d65-b607-50e9ed036806 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:23.857724 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4515527-1aeb-43f9-a709-18f6fb507827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:23.858002 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4515527-1aeb-43f9-a709-18f6fb507827 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:23.858274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4515527-1aeb-43f9-a709-18f6fb507827 admin admin] PATH_INFO: `/v3/users/f3735a60edb44f459e9fdb91aeef47fb` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:23.859397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4515527-1aeb-43f9-a709-18f6fb507827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3735a60edb44f459e9fdb91aeef47fb)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:23.862184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4515527-1aeb-43f9-a709-18f6fb507827 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:23.868035 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1045/2105] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:24.053064 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e4515527-1aeb-43f9-a709-18f6fb507827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3735a60edb44f459e9fdb91aeef47fb'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:24.060183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e4515527-1aeb-43f9-a709-18f6fb507827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3735a60edb44f459e9fdb91aeef47fb'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:24.063939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:24.124693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e4515527-1aeb-43f9-a709-18f6fb507827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3735a60edb44f459e9fdb91aeef47fb'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:24.126509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e4515527-1aeb-43f9-a709-18f6fb507827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3735a60edb44f459e9fdb91aeef47fb'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:24.143145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e4515527-1aeb-43f9-a709-18f6fb507827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3735a60edb44f459e9fdb91aeef47fb'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:24.150959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1061/2106] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:23 2022] DELETE /identity/v3/users/f3735a60edb44f459e9fdb91aeef47fb => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:24.155177 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968007c0-5630-4740-93d7-bb1ee922d5af None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:24.155474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968007c0-5630-4740-93d7-bb1ee922d5af None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:24.155752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968007c0-5630-4740-93d7-bb1ee922d5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:24.177311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-968007c0-5630-4740-93d7-bb1ee922d5af None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:24.179169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-968007c0-5630-4740-93d7-bb1ee922d5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:24.248286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1046/2107] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:24.261104 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4515527-1aeb-43f9-a709-18f6fb507827 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:24.262508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e4515527-1aeb-43f9-a709-18f6fb507827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:24.316095 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968007c0-5630-4740-93d7-bb1ee922d5af None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:24.317397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-968007c0-5630-4740-93d7-bb1ee922d5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:24.320366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4515527-1aeb-43f9-a709-18f6fb507827 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:24.324727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300152c7-a163-4d9b-b8a3-2fc85efbc6f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:24.325560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300152c7-a163-4d9b-b8a3-2fc85efbc6f8 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:24.325947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300152c7-a163-4d9b-b8a3-2fc85efbc6f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:24.326131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300152c7-a163-4d9b-b8a3-2fc85efbc6f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:24.330420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968007c0-5630-4740-93d7-bb1ee922d5af None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:24.335151 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414ff5b-cf96-489f-8eab-c5bc39421c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:24.335776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414ff5b-cf96-489f-8eab-c5bc39421c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:24.336055 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414ff5b-cf96-489f-8eab-c5bc39421c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:24.336327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414ff5b-cf96-489f-8eab-c5bc39421c06 admin admin] PATH_INFO: `/v3/projects/5cf5fee9c60d4cefb9d2ef03b313fb4a` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:24.337536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414ff5b-cf96-489f-8eab-c5bc39421c06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cf5fee9c60d4cefb9d2ef03b313fb4a)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:24.340043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414ff5b-cf96-489f-8eab-c5bc39421c06 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:24.342348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300152c7-a163-4d9b-b8a3-2fc85efbc6f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:24.343839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300152c7-a163-4d9b-b8a3-2fc85efbc6f8 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:24.354442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1062/2108] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:24.429219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:24.436772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-6414ff5b-cf96-489f-8eab-c5bc39421c06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cf5fee9c60d4cefb9d2ef03b313fb4a', 'request_id': 'req-6414ff5b-cf96-489f-8eab-c5bc39421c06'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:24.443307 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-6414ff5b-cf96-489f-8eab-c5bc39421c06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cf5fee9c60d4cefb9d2ef03b313fb4a', 'request_id': 'req-6414ff5b-cf96-489f-8eab-c5bc39421c06'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:24.490473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-6414ff5b-cf96-489f-8eab-c5bc39421c06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cf5fee9c60d4cefb9d2ef03b313fb4a', 'request_id': 'req-6414ff5b-cf96-489f-8eab-c5bc39421c06'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:24.501631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-6414ff5b-cf96-489f-8eab-c5bc39421c06 admin admin] The token cache is being invalidate because project 5cf5fee9c60d4cefb9d2ef03b313fb4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:24.502036 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-6414ff5b-cf96-489f-8eab-c5bc39421c06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:24.502856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-6414ff5b-cf96-489f-8eab-c5bc39421c06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:24.504280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1047/2109] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:24 2022] DELETE /identity/v3/projects/5cf5fee9c60d4cefb9d2ef03b313fb4a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:24.509307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05fb2d-01c4-418b-9110-cd3bec669e57 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:24.509585 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05fb2d-01c4-418b-9110-cd3bec669e57 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:24.509866 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05fb2d-01c4-418b-9110-cd3bec669e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:24.542737 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-fb05fb2d-01c4-418b-9110-cd3bec669e57 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:24.544161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-fb05fb2d-01c4-418b-9110-cd3bec669e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:24.634798 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1063/2110] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:24.642485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86916d51-4303-467b-8300-85c3529b6973 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:24.642989 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86916d51-4303-467b-8300-85c3529b6973 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:24.643235 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86916d51-4303-467b-8300-85c3529b6973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:24.665503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-86916d51-4303-467b-8300-85c3529b6973 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:24.666826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-86916d51-4303-467b-8300-85c3529b6973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:24.675770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1048/2111] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:24.680237 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05fb2d-01c4-418b-9110-cd3bec669e57 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:24.691210 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05fb2d-01c4-418b-9110-cd3bec669e57 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:24.695992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f101b2a-a26d-4896-9cde-18af899a7f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:24.696590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101b2a-a26d-4896-9cde-18af899a7f26 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:24.696901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101b2a-a26d-4896-9cde-18af899a7f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:24.697176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101b2a-a26d-4896-9cde-18af899a7f26 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:24.697853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f101b2a-a26d-4896-9cde-18af899a7f26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:53:24.698160 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f101b2a-a26d-4896-9cde-18af899a7f26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:24.699804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f101b2a-a26d-4896-9cde-18af899a7f26 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:24.712915 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1064/2112] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:53:24.717415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86916d51-4303-467b-8300-85c3529b6973 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:24.728188 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86916d51-4303-467b-8300-85c3529b6973 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:24.733024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04ba2b-94dc-4f30-96c4-7ce56928daf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:24.733625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04ba2b-94dc-4f30-96c4-7ce56928daf2 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:24.736830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04ba2b-94dc-4f30-96c4-7ce56928daf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:24.736830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04ba2b-94dc-4f30-96c4-7ce56928daf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:24.736830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04ba2b-94dc-4f30-96c4-7ce56928daf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:24.737318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04ba2b-94dc-4f30-96c4-7ce56928daf2 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:24.783844 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f101b2a-a26d-4896-9cde-18af899a7f26 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:24.785592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5f101b2a-a26d-4896-9cde-18af899a7f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:24.803184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f101b2a-a26d-4896-9cde-18af899a7f26 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:24.804578 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:24.805242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:24.805527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:24.805834 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] PATH_INFO: `/v3/projects/6c92d3a55c5f4db69e29c2043e2e0765` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:24.806994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c92d3a55c5f4db69e29c2043e2e0765)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:24.808967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:24.886469 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1049/2113] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:24 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:53:24.890702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04ba2b-94dc-4f30-96c4-7ce56928daf2 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:24.959957 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04ba2b-94dc-4f30-96c4-7ce56928daf2 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:24.976066 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c92d3a55c5f4db69e29c2043e2e0765', 'request_id': 'req-5fcee875-0845-41e6-8dee-c23f57816e18'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:24.984751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c92d3a55c5f4db69e29c2043e2e0765', 'request_id': 'req-5fcee875-0845-41e6-8dee-c23f57816e18'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:24.991043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c92d3a55c5f4db69e29c2043e2e0765', 'request_id': 'req-5fcee875-0845-41e6-8dee-c23f57816e18'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:24.992881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:25.020395 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db3429-1af3-446c-aca8-26d5a643d1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:25.021068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db3429-1af3-446c-aca8-26d5a643d1a4 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:25.021378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db3429-1af3-446c-aca8-26d5a643d1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:25.021651 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db3429-1af3-446c-aca8-26d5a643d1a4 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:25.022299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db3429-1af3-446c-aca8-26d5a643d1a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:25.024225 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db3429-1af3-446c-aca8-26d5a643d1a4 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:25.067417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] The token cache is being invalidate because project 6c92d3a55c5f4db69e29c2043e2e0765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:25.067711 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:25.068407 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:25.069982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1065/2114] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:24 2022] DELETE /identity/v3/projects/6c92d3a55c5f4db69e29c2043e2e0765 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:25.073851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:25.075177 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:25.088411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcee875-0845-41e6-8dee-c23f57816e18 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:25.093135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71745939-e014-4f90-9aac-dc980eb9d84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:25.093752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71745939-e014-4f90-9aac-dc980eb9d84c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:25.094033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71745939-e014-4f90-9aac-dc980eb9d84c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:25.094307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71745939-e014-4f90-9aac-dc980eb9d84c admin admin] PATH_INFO: `/v3/users/c01fba8d0aa8409097e5f6dec6c0ccc3` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:25.095433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71745939-e014-4f90-9aac-dc980eb9d84c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c01fba8d0aa8409097e5f6dec6c0ccc3)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:25.097379 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71745939-e014-4f90-9aac-dc980eb9d84c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:25.136699 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1050/2115] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:24 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:53:25.140535 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db3429-1af3-446c-aca8-26d5a643d1a4 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:25.142581 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c1db3429-1af3-446c-aca8-26d5a643d1a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:25.154572 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db3429-1af3-446c-aca8-26d5a643d1a4 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:25.159368 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f61e9-7776-480b-b61d-948f6b3e44e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:25.159949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f61e9-7776-480b-b61d-948f6b3e44e9 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:25.160231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f61e9-7776-480b-b61d-948f6b3e44e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:25.160507 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f61e9-7776-480b-b61d-948f6b3e44e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:25.161213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f61e9-7776-480b-b61d-948f6b3e44e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:25.162797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f61e9-7776-480b-b61d-948f6b3e44e9 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:25.172424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1051/2116] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:53:25.176289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f61e9-7776-480b-b61d-948f6b3e44e9 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:25.185396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f61e9-7776-480b-b61d-948f6b3e44e9 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:25.190803 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d6265-2e3a-460f-905a-d1ed0790adda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:25.191362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d6265-2e3a-460f-905a-d1ed0790adda admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:25.191657 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d6265-2e3a-460f-905a-d1ed0790adda admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:25.192013 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d6265-2e3a-460f-905a-d1ed0790adda admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:25.198869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d6265-2e3a-460f-905a-d1ed0790adda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:25.198869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d6265-2e3a-460f-905a-d1ed0790adda admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:25.205683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1052/2117] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:53:25.209494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d6265-2e3a-460f-905a-d1ed0790adda admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:25.219002 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d6265-2e3a-460f-905a-d1ed0790adda admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:25.223718 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083af14a-8a0f-4092-929e-a8c48b065450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:25.224342 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083af14a-8a0f-4092-929e-a8c48b065450 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:25.227604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083af14a-8a0f-4092-929e-a8c48b065450 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:25.227604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083af14a-8a0f-4092-929e-a8c48b065450 admin admin] PATH_INFO: `/v3/projects/b3bf9d51878b413aac99dbcf7f969643/users/a238c069d143404fa73cf1f955fba883/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:25.236120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:25.240039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083af14a-8a0f-4092-929e-a8c48b065450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3bf9d51878b413aac99dbcf7f969643, user_id=a238c069d143404fa73cf1f955fba883, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:25.242717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083af14a-8a0f-4092-929e-a8c48b065450 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:25.323176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-71745939-e014-4f90-9aac-dc980eb9d84c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c01fba8d0aa8409097e5f6dec6c0ccc3'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:25.331183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-71745939-e014-4f90-9aac-dc980eb9d84c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c01fba8d0aa8409097e5f6dec6c0ccc3'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:25.359350 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1053/2118] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:25 2022] PUT /identity/v3/projects/b3bf9d51878b413aac99dbcf7f969643/users/a238c069d143404fa73cf1f955fba883/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:25.370898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862d73e-8e1b-4843-849d-7c91eb4f4990 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:25.371240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862d73e-8e1b-4843-849d-7c91eb4f4990 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:25.391144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862d73e-8e1b-4843-849d-7c91eb4f4990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:25.405052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d862d73e-8e1b-4843-849d-7c91eb4f4990 None None] MFA Rules not processed for user `a238c069d143404fa73cf1f955fba883`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:25.406314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d862d73e-8e1b-4843-849d-7c91eb4f4990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:25.450206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-71745939-e014-4f90-9aac-dc980eb9d84c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c01fba8d0aa8409097e5f6dec6c0ccc3'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:25.450836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-71745939-e014-4f90-9aac-dc980eb9d84c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c01fba8d0aa8409097e5f6dec6c0ccc3'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:25.463884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-71745939-e014-4f90-9aac-dc980eb9d84c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c01fba8d0aa8409097e5f6dec6c0ccc3'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:25.470335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1066/2119] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:25 2022] DELETE /identity/v3/users/c01fba8d0aa8409097e5f6dec6c0ccc3 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:25.474009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1054/2120] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:53:25.478282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71745939-e014-4f90-9aac-dc980eb9d84c admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:25.479706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-71745939-e014-4f90-9aac-dc980eb9d84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:25.550553 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71745939-e014-4f90-9aac-dc980eb9d84c admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:25.555244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595b0d6-9239-47fe-8bac-8c7caeab0ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:25.555853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595b0d6-9239-47fe-8bac-8c7caeab0ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:25.556136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595b0d6-9239-47fe-8bac-8c7caeab0ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:25.556416 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595b0d6-9239-47fe-8bac-8c7caeab0ca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:25.557149 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595b0d6-9239-47fe-8bac-8c7caeab0ca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:25.559052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595b0d6-9239-47fe-8bac-8c7caeab0ca4 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:25.651949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1067/2121] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:25 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:25.656168 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862d73e-8e1b-4843-849d-7c91eb4f4990 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:25.729128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862d73e-8e1b-4843-849d-7c91eb4f4990 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:25.795603 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc82f1-8e9e-4df6-b6e8-e4d07a627720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:25.796335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc82f1-8e9e-4df6-b6e8-e4d07a627720 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:25.796778 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc82f1-8e9e-4df6-b6e8-e4d07a627720 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:25.797115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc82f1-8e9e-4df6-b6e8-e4d07a627720 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:25.798343 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc82f1-8e9e-4df6-b6e8-e4d07a627720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:25.800896 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc82f1-8e9e-4df6-b6e8-e4d07a627720 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:25.936839 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1055/2122] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:25 2022] POST /identity/v3/users => generated 288 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:53:25.937604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595b0d6-9239-47fe-8bac-8c7caeab0ca4 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:25.939973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5595b0d6-9239-47fe-8bac-8c7caeab0ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:25.940638 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc82f1-8e9e-4df6-b6e8-e4d07a627720 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:25.951017 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc82f1-8e9e-4df6-b6e8-e4d07a627720 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:25.954331 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595b0d6-9239-47fe-8bac-8c7caeab0ca4 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:25.956595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576e072-6bf6-4586-bf71-15b94152aaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:25.957309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576e072-6bf6-4586-bf71-15b94152aaa8 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:25.957663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576e072-6bf6-4586-bf71-15b94152aaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:25.958007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576e072-6bf6-4586-bf71-15b94152aaa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:25.958714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576e072-6bf6-4586-bf71-15b94152aaa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:25.960548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576e072-6bf6-4586-bf71-15b94152aaa8 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:25.963111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686a577-ccd3-4f7e-8feb-ce9318abf068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:25.963826 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686a577-ccd3-4f7e-8feb-ce9318abf068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:25.964209 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686a577-ccd3-4f7e-8feb-ce9318abf068 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:25.964561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686a577-ccd3-4f7e-8feb-ce9318abf068 admin admin] PATH_INFO: `/v3/projects/f1b1b3574cd84b1c80511df952b62f38` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:25.965811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686a577-ccd3-4f7e-8feb-ce9318abf068 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b1b3574cd84b1c80511df952b62f38)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:25.967809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686a577-ccd3-4f7e-8feb-ce9318abf068 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:25.975080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1056/2123] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:53:25.975977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576e072-6bf6-4586-bf71-15b94152aaa8 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:25.987804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576e072-6bf6-4586-bf71-15b94152aaa8 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:25.992868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a706b-895a-4777-ae01-cc81af49c218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:25.993468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a706b-895a-4777-ae01-cc81af49c218 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:25.993749 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a706b-895a-4777-ae01-cc81af49c218 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:25.994494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a706b-895a-4777-ae01-cc81af49c218 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:25.994643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a706b-895a-4777-ae01-cc81af49c218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:25.996233 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a706b-895a-4777-ae01-cc81af49c218 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:26.011941 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1057/2124] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:26.011941 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a706b-895a-4777-ae01-cc81af49c218 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:26.021290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a706b-895a-4777-ae01-cc81af49c218 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:26.026283 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd57a4e-a4d4-4258-9ba9-32713a490029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:26.026959 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd57a4e-a4d4-4258-9ba9-32713a490029 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.029008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd57a4e-a4d4-4258-9ba9-32713a490029 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.029008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd57a4e-a4d4-4258-9ba9-32713a490029 admin admin] PATH_INFO: `/v3/projects/f28c9de2c1e5451aa1ad76d3b18be5bb/users/8134ef40af76455d8d7a334957fc1f83/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.044295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd57a4e-a4d4-4258-9ba9-32713a490029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f28c9de2c1e5451aa1ad76d3b18be5bb, user_id=8134ef40af76455d8d7a334957fc1f83, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:26.047074 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd57a4e-a4d4-4258-9ba9-32713a490029 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:26.056419 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:INSERT=3,keystone:DELETE=14,keystone:SELECT=83 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:26.060460 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e686a577-ccd3-4f7e-8feb-ce9318abf068 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b1b3574cd84b1c80511df952b62f38', 'request_id': 'req-e686a577-ccd3-4f7e-8feb-ce9318abf068'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:26.069352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e686a577-ccd3-4f7e-8feb-ce9318abf068 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b1b3574cd84b1c80511df952b62f38', 'request_id': 'req-e686a577-ccd3-4f7e-8feb-ce9318abf068'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:26.074886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e686a577-ccd3-4f7e-8feb-ce9318abf068 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b1b3574cd84b1c80511df952b62f38', 'request_id': 'req-e686a577-ccd3-4f7e-8feb-ce9318abf068'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:26.124780 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1058/2125] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:26 2022] PUT /identity/v3/projects/f28c9de2c1e5451aa1ad76d3b18be5bb/users/8134ef40af76455d8d7a334957fc1f83/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:26.135083 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974a9a8-5dd8-4768-ab4a-cbd3cc0f3491 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.135446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974a9a8-5dd8-4768-ab4a-cbd3cc0f3491 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.135838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974a9a8-5dd8-4768-ab4a-cbd3cc0f3491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.148546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e686a577-ccd3-4f7e-8feb-ce9318abf068 admin admin] The token cache is being invalidate because project f1b1b3574cd84b1c80511df952b62f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:26.149041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e686a577-ccd3-4f7e-8feb-ce9318abf068 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:26.149965 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e686a577-ccd3-4f7e-8feb-ce9318abf068 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:26.164826 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1068/2126] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:25 2022] DELETE /identity/v3/projects/f1b1b3574cd84b1c80511df952b62f38 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:26.164826 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28484ac0-0835-4bf1-9902-223406e57d87 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.164826 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28484ac0-0835-4bf1-9902-223406e57d87 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.164826 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28484ac0-0835-4bf1-9902-223406e57d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.167476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-f974a9a8-5dd8-4768-ab4a-cbd3cc0f3491 None None] MFA Rules not processed for user `8134ef40af76455d8d7a334957fc1f83`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:26.168717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f974a9a8-5dd8-4768-ab4a-cbd3cc0f3491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:26.180580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-28484ac0-0835-4bf1-9902-223406e57d87 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:26.181721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-28484ac0-0835-4bf1-9902-223406e57d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:26.232822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1059/2127] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:26.237478 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974a9a8-5dd8-4768-ab4a-cbd3cc0f3491 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:26.238782 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f974a9a8-5dd8-4768-ab4a-cbd3cc0f3491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:26.242049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1069/2128] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:26.249518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84a72d-dc6b-4ace-b964-12e533ad8d3e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.249832 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84a72d-dc6b-4ace-b964-12e533ad8d3e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.250052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84a72d-dc6b-4ace-b964-12e533ad8d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.252696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:26.254022 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974a9a8-5dd8-4768-ab4a-cbd3cc0f3491 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:26.258677 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3de41-7eb8-4373-855d-e8f7cf6323b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:26.261729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3de41-7eb8-4373-855d-e8f7cf6323b3 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.261729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3de41-7eb8-4373-855d-e8f7cf6323b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.261729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3de41-7eb8-4373-855d-e8f7cf6323b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.261729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3de41-7eb8-4373-855d-e8f7cf6323b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:26.262484 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3de41-7eb8-4373-855d-e8f7cf6323b3 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:26.270638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9f84a72d-dc6b-4ace-b964-12e533ad8d3e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:26.271878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9f84a72d-dc6b-4ace-b964-12e533ad8d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:26.281726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1070/2129] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:53:26.287002 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84a72d-dc6b-4ace-b964-12e533ad8d3e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:26.295929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84a72d-dc6b-4ace-b964-12e533ad8d3e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:26.301226 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb869a2f-a3b1-4c5f-a060-d129e34a188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:26.301807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb869a2f-a3b1-4c5f-a060-d129e34a188d admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.302095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb869a2f-a3b1-4c5f-a060-d129e34a188d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.302371 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb869a2f-a3b1-4c5f-a060-d129e34a188d admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.303004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb869a2f-a3b1-4c5f-a060-d129e34a188d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:53:26.303306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb869a2f-a3b1-4c5f-a060-d129e34a188d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:26.304972 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb869a2f-a3b1-4c5f-a060-d129e34a188d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:26.317350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1071/2130] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:53:26.323636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb869a2f-a3b1-4c5f-a060-d129e34a188d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:26.333719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb869a2f-a3b1-4c5f-a060-d129e34a188d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:26.338301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1060/2131] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:26 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:26.338560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156831b-c4aa-4c0c-ad96-d0430cf14e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:26.339215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156831b-c4aa-4c0c-ad96-d0430cf14e82 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.339501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156831b-c4aa-4c0c-ad96-d0430cf14e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.339819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156831b-c4aa-4c0c-ad96-d0430cf14e82 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.340447 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156831b-c4aa-4c0c-ad96-d0430cf14e82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:26.342468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3de41-7eb8-4373-855d-e8f7cf6323b3 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:26.343358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156831b-c4aa-4c0c-ad96-d0430cf14e82 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:26.404547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3de41-7eb8-4373-855d-e8f7cf6323b3 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:26.409185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55705a-15fd-4b28-843c-5f1f05e581d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:26.409783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55705a-15fd-4b28-843c-5f1f05e581d5 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.410066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55705a-15fd-4b28-843c-5f1f05e581d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.410342 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55705a-15fd-4b28-843c-5f1f05e581d5 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.410993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55705a-15fd-4b28-843c-5f1f05e581d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:26.412940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55705a-15fd-4b28-843c-5f1f05e581d5 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:26.418178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1072/2132] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:26 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:53:26.421480 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156831b-c4aa-4c0c-ad96-d0430cf14e82 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:26.496532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1061/2133] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:26 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:26.499374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156831b-c4aa-4c0c-ad96-d0430cf14e82 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:26.502602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55705a-15fd-4b28-843c-5f1f05e581d5 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:26.506967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf50bc0-a590-45c8-ac0e-073abf33859d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:26.507678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf50bc0-a590-45c8-ac0e-073abf33859d admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.508161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf50bc0-a590-45c8-ac0e-073abf33859d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.508545 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf50bc0-a590-45c8-ac0e-073abf33859d admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.509336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf50bc0-a590-45c8-ac0e-073abf33859d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:26.511485 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf50bc0-a590-45c8-ac0e-073abf33859d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:26.517297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55705a-15fd-4b28-843c-5f1f05e581d5 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:26.528318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c79d7-56c5-43a5-a265-4fb285e105a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:26.529868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c79d7-56c5-43a5-a265-4fb285e105a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.530200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c79d7-56c5-43a5-a265-4fb285e105a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.530476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c79d7-56c5-43a5-a265-4fb285e105a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.531457 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c79d7-56c5-43a5-a265-4fb285e105a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:26.533651 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c79d7-56c5-43a5-a265-4fb285e105a8 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:26.555466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1062/2134] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:26.559516 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c79d7-56c5-43a5-a265-4fb285e105a8 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:26.570434 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c79d7-56c5-43a5-a265-4fb285e105a8 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:26.576247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa80d2e-4c2e-4e08-b14d-591b980d4a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:26.576945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa80d2e-4c2e-4e08-b14d-591b980d4a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.577233 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa80d2e-4c2e-4e08-b14d-591b980d4a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.577556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa80d2e-4c2e-4e08-b14d-591b980d4a0c admin admin] PATH_INFO: `/v3/projects/6aea865fe10f441bb7b41396af8750e7/users/e370d55247cd4e48a057d1f57c888ecf/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.592464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:26.596517 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa80d2e-4c2e-4e08-b14d-591b980d4a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aea865fe10f441bb7b41396af8750e7, user_id=e370d55247cd4e48a057d1f57c888ecf, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:26.599304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa80d2e-4c2e-4e08-b14d-591b980d4a0c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:26.617348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1073/2135] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:26 2022] POST /identity/v3/users => generated 294 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:53:26.621163 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf50bc0-a590-45c8-ac0e-073abf33859d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:26.634821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf50bc0-a590-45c8-ac0e-073abf33859d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:26.643200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce0c7d-a8a7-4b6c-8782-814d26bc1bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:26.644474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce0c7d-a8a7-4b6c-8782-814d26bc1bef admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.645408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce0c7d-a8a7-4b6c-8782-814d26bc1bef admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.645882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce0c7d-a8a7-4b6c-8782-814d26bc1bef admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.646930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce0c7d-a8a7-4b6c-8782-814d26bc1bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:26.648552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce0c7d-a8a7-4b6c-8782-814d26bc1bef admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:26.660754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1074/2136] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:53:26.664090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1063/2137] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:26 2022] PUT /identity/v3/projects/6aea865fe10f441bb7b41396af8750e7/users/e370d55247cd4e48a057d1f57c888ecf/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:26.667507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce0c7d-a8a7-4b6c-8782-814d26bc1bef admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:26.668249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa80d2e-4c2e-4e08-b14d-591b980d4a0c admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:26.678461 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce0c7d-a8a7-4b6c-8782-814d26bc1bef admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:26.683513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:26.740759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8853c-e1d0-4a90-bc6a-a8970b35e3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:26.746001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa80d2e-4c2e-4e08-b14d-591b980d4a0c admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:26.746566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8853c-e1d0-4a90-bc6a-a8970b35e3ab admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.746566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8853c-e1d0-4a90-bc6a-a8970b35e3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.746566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8853c-e1d0-4a90-bc6a-a8970b35e3ab admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.746566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8853c-e1d0-4a90-bc6a-a8970b35e3ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:26.746566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8853c-e1d0-4a90-bc6a-a8970b35e3ab admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:26.746953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc956af2-b75a-4f70-98e9-c86d838d186a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:26.747611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc956af2-b75a-4f70-98e9-c86d838d186a admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.747902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc956af2-b75a-4f70-98e9-c86d838d186a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.748174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc956af2-b75a-4f70-98e9-c86d838d186a admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.749706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc956af2-b75a-4f70-98e9-c86d838d186a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:26.751979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc956af2-b75a-4f70-98e9-c86d838d186a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:26.757605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1075/2138] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:53:26.763481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8853c-e1d0-4a90-bc6a-a8970b35e3ab admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:26.772218 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1064/2139] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:26.775741 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8853c-e1d0-4a90-bc6a-a8970b35e3ab admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:26.781216 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9e702-3e51-465c-8876-3b56aaa798f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:26.782710 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9e702-3e51-465c-8876-3b56aaa798f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.783111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9e702-3e51-465c-8876-3b56aaa798f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.783494 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9e702-3e51-465c-8876-3b56aaa798f4 admin admin] PATH_INFO: `/v3/projects/a49b52f2a40744eaab8ac62e6bd96807/users/75578ddbb9b74379a3a8ec3ba989b1e8/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.790474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc956af2-b75a-4f70-98e9-c86d838d186a admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:26.800302 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9e702-3e51-465c-8876-3b56aaa798f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49b52f2a40744eaab8ac62e6bd96807, user_id=75578ddbb9b74379a3a8ec3ba989b1e8, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:26.805550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9e702-3e51-465c-8876-3b56aaa798f4 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:26.809175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc956af2-b75a-4f70-98e9-c86d838d186a admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:26.815803 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f6c99-3bbf-4dfb-9857-17345732dfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:26.816478 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f6c99-3bbf-4dfb-9857-17345732dfdc admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.816792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f6c99-3bbf-4dfb-9857-17345732dfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.817072 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f6c99-3bbf-4dfb-9857-17345732dfdc admin admin] PATH_INFO: `/v3/projects/6aea865fe10f441bb7b41396af8750e7/users/e370d55247cd4e48a057d1f57c888ecf/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.819007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f6c99-3bbf-4dfb-9857-17345732dfdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aea865fe10f441bb7b41396af8750e7, user_id=e370d55247cd4e48a057d1f57c888ecf, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:26.821941 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f6c99-3bbf-4dfb-9857-17345732dfdc admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:26.886822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1065/2140] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:26 2022] PUT /identity/v3/projects/6aea865fe10f441bb7b41396af8750e7/users/e370d55247cd4e48a057d1f57c888ecf/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:26.898104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca383533-8f39-4338-a151-5f170005fe41 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.898390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca383533-8f39-4338-a151-5f170005fe41 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.898668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca383533-8f39-4338-a151-5f170005fe41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.926646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-ca383533-8f39-4338-a151-5f170005fe41 None None] MFA Rules not processed for user `e370d55247cd4e48a057d1f57c888ecf`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:26.927875 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ca383533-8f39-4338-a151-5f170005fe41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:26.965128 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1076/2141] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:26 2022] PUT /identity/v3/projects/a49b52f2a40744eaab8ac62e6bd96807/users/75578ddbb9b74379a3a8ec3ba989b1e8/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:26.975631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a9606-10d9-4ef4-a7fe-d4da6bd27565 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:26.976152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a9606-10d9-4ef4-a7fe-d4da6bd27565 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:26.976429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a9606-10d9-4ef4-a7fe-d4da6bd27565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:26.999457 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1066/2142] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:53:27.005564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-165a9606-10d9-4ef4-a7fe-d4da6bd27565 None None] MFA Rules not processed for user `75578ddbb9b74379a3a8ec3ba989b1e8`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:27.006788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-165a9606-10d9-4ef4-a7fe-d4da6bd27565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:27.012014 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca383533-8f39-4338-a151-5f170005fe41 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:27.013428 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ca383533-8f39-4338-a151-5f170005fe41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:27.072371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca383533-8f39-4338-a151-5f170005fe41 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:27.073324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1077/2143] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:27.077998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a9606-10d9-4ef4-a7fe-d4da6bd27565 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:27.085200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:27.124687 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3a2c6-c0d8-4785-a04b-17de87879e5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:27.125560 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3a2c6-c0d8-4785-a04b-17de87879e5f service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:27.126986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3a2c6-c0d8-4785-a04b-17de87879e5f service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:27.127780 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3a2c6-c0d8-4785-a04b-17de87879e5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:27.129743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-63d3a2c6-c0d8-4785-a04b-17de87879e5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:27.143169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a9606-10d9-4ef4-a7fe-d4da6bd27565 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:27.147726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b29b5b-aa31-49fe-9379-a97cc49634a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:27.148329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b29b5b-aa31-49fe-9379-a97cc49634a3 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:27.148610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b29b5b-aa31-49fe-9379-a97cc49634a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:27.148917 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b29b5b-aa31-49fe-9379-a97cc49634a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:27.149541 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b29b5b-aa31-49fe-9379-a97cc49634a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:27.151448 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b29b5b-aa31-49fe-9379-a97cc49634a3 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:27.167348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:27.192764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3a2c6-c0d8-4785-a04b-17de87879e5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:27.194229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3a2c6-c0d8-4785-a04b-17de87879e5f service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:27.203260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1067/2144] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:53:27.237786 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1078/2145] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:27 2022] POST /identity/v3/projects => generated 389 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:53:27.241697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3a2c6-c0d8-4785-a04b-17de87879e5f service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:27.255390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3a2c6-c0d8-4785-a04b-17de87879e5f service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:27.331287 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0043623-0e53-429b-9cd7-cf2c54791113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:27.331930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0043623-0e53-429b-9cd7-cf2c54791113 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:27.332191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0043623-0e53-429b-9cd7-cf2c54791113 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:27.332468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0043623-0e53-429b-9cd7-cf2c54791113 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:27.333213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0043623-0e53-429b-9cd7-cf2c54791113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:27.335310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0043623-0e53-429b-9cd7-cf2c54791113 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:27.425428 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1068/2146] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:27 2022] POST /identity/v3/users => generated 301 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:27.427431 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b29b5b-aa31-49fe-9379-a97cc49634a3 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:27.438118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b29b5b-aa31-49fe-9379-a97cc49634a3 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:27.444499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33adadfa-c21b-4dd8-bc4b-7f183da26754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:27.445244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33adadfa-c21b-4dd8-bc4b-7f183da26754 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:27.445535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33adadfa-c21b-4dd8-bc4b-7f183da26754 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:27.446386 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33adadfa-c21b-4dd8-bc4b-7f183da26754 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:27.446727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33adadfa-c21b-4dd8-bc4b-7f183da26754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:27.448255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33adadfa-c21b-4dd8-bc4b-7f183da26754 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:27.459900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1079/2147] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:27.464192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0043623-0e53-429b-9cd7-cf2c54791113 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:27.474601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0043623-0e53-429b-9cd7-cf2c54791113 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:27.479547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f768dd-801d-4873-86b8-b067914f867a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:27.480202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f768dd-801d-4873-86b8-b067914f867a admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:27.480485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f768dd-801d-4873-86b8-b067914f867a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:27.480789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f768dd-801d-4873-86b8-b067914f867a admin admin] PATH_INFO: `/v3/projects/3b5664710df74519a3db974eddf4e197/users/2880d6868dec4335a538705159774349/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:27.497292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f768dd-801d-4873-86b8-b067914f867a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b5664710df74519a3db974eddf4e197, user_id=2880d6868dec4335a538705159774349, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:27.500035 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f768dd-801d-4873-86b8-b067914f867a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:27.565803 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1069/2148] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:27 2022] PUT /identity/v3/projects/3b5664710df74519a3db974eddf4e197/users/2880d6868dec4335a538705159774349/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:27.569851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33adadfa-c21b-4dd8-bc4b-7f183da26754 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:27.580762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33adadfa-c21b-4dd8-bc4b-7f183da26754 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:27.652015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bb290-b591-4b50-bc39-ae3ee93d1bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:27.652833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bb290-b591-4b50-bc39-ae3ee93d1bda admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:27.653152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bb290-b591-4b50-bc39-ae3ee93d1bda admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:27.653426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bb290-b591-4b50-bc39-ae3ee93d1bda admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:27.659955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bb290-b591-4b50-bc39-ae3ee93d1bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:27.659955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bb290-b591-4b50-bc39-ae3ee93d1bda admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:27.670154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1080/2149] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:27.674671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f768dd-801d-4873-86b8-b067914f867a admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:27.686800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f768dd-801d-4873-86b8-b067914f867a admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:27.693181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0d297-c026-49e5-bd67-ea519ddd2d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:27.693987 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0d297-c026-49e5-bd67-ea519ddd2d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:27.694336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0d297-c026-49e5-bd67-ea519ddd2d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:27.694772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0d297-c026-49e5-bd67-ea519ddd2d7c admin admin] PATH_INFO: `/v3/projects/3b5664710df74519a3db974eddf4e197/users/2880d6868dec4335a538705159774349/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:27.697852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0d297-c026-49e5-bd67-ea519ddd2d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b5664710df74519a3db974eddf4e197, user_id=2880d6868dec4335a538705159774349, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:27.700905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0d297-c026-49e5-bd67-ea519ddd2d7c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:27.718137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1070/2150] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:27 2022] PUT /identity/v3/projects/3b5664710df74519a3db974eddf4e197/users/2880d6868dec4335a538705159774349/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:27.731370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d9e04-cf79-4f4c-a206-986b4f7a17c3 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:27.731710 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d9e04-cf79-4f4c-a206-986b4f7a17c3 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:27.732018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d9e04-cf79-4f4c-a206-986b4f7a17c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:27.777414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-251d9e04-cf79-4f4c-a206-986b4f7a17c3 None None] MFA Rules not processed for user `2880d6868dec4335a538705159774349`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:27.778659 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-251d9e04-cf79-4f4c-a206-986b4f7a17c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:27.913907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1081/2151] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:27 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:27.939202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0d297-c026-49e5-bd67-ea519ddd2d7c admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:27.939202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-a5f0d297-c026-49e5-bd67-ea519ddd2d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:27.983051 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:28.035518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0d297-c026-49e5-bd67-ea519ddd2d7c admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:28.040802 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331c836-a96f-44d5-b5de-a88860ef6a94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:28.041595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331c836-a96f-44d5-b5de-a88860ef6a94 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:28.042069 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331c836-a96f-44d5-b5de-a88860ef6a94 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:28.042333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331c836-a96f-44d5-b5de-a88860ef6a94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:28.055618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331c836-a96f-44d5-b5de-a88860ef6a94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:28.057308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331c836-a96f-44d5-b5de-a88860ef6a94 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:28.067920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1071/2152] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:27 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:28.216759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d9e04-cf79-4f4c-a206-986b4f7a17c3 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:28.283774 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d9e04-cf79-4f4c-a206-986b4f7a17c3 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:28.288296 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ba93f-4958-485b-9188-e2ebc6d38ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:28.293178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ba93f-4958-485b-9188-e2ebc6d38ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:28.293178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ba93f-4958-485b-9188-e2ebc6d38ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:28.293178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ba93f-4958-485b-9188-e2ebc6d38ffe admin admin] PATH_INFO: `/v3/users/75578ddbb9b74379a3a8ec3ba989b1e8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:28.293178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ba93f-4958-485b-9188-e2ebc6d38ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=75578ddbb9b74379a3a8ec3ba989b1e8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:28.293178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ba93f-4958-485b-9188-e2ebc6d38ffe admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:28.298243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:28.568505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ec9ba93f-4958-485b-9188-e2ebc6d38ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75578ddbb9b74379a3a8ec3ba989b1e8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:28.575698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ec9ba93f-4958-485b-9188-e2ebc6d38ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75578ddbb9b74379a3a8ec3ba989b1e8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:28.695656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ec9ba93f-4958-485b-9188-e2ebc6d38ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75578ddbb9b74379a3a8ec3ba989b1e8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:28.696284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ec9ba93f-4958-485b-9188-e2ebc6d38ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75578ddbb9b74379a3a8ec3ba989b1e8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:28.708914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ec9ba93f-4958-485b-9188-e2ebc6d38ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75578ddbb9b74379a3a8ec3ba989b1e8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:28.714765 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1082/2153] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:28 2022] DELETE /identity/v3/users/75578ddbb9b74379a3a8ec3ba989b1e8 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:28.719370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc3215-f7c8-410b-8fa8-3c4ff97ed225 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:28.719965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc3215-f7c8-410b-8fa8-3c4ff97ed225 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:28.719965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc3215-f7c8-410b-8fa8-3c4ff97ed225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:28.740068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e1fc3215-f7c8-410b-8fa8-3c4ff97ed225 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:28.741293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e1fc3215-f7c8-410b-8fa8-3c4ff97ed225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:28.800977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1072/2154] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:53:28.810919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ba93f-4958-485b-9188-e2ebc6d38ffe admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:28.812341 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ec9ba93f-4958-485b-9188-e2ebc6d38ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:28.870902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ba93f-4958-485b-9188-e2ebc6d38ffe admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:28.874543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29771daf-cd3e-4403-8b11-9c7e5ce32655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:28.875131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29771daf-cd3e-4403-8b11-9c7e5ce32655 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:28.875408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29771daf-cd3e-4403-8b11-9c7e5ce32655 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:28.875683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29771daf-cd3e-4403-8b11-9c7e5ce32655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:28.884784 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29771daf-cd3e-4403-8b11-9c7e5ce32655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:28.886201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29771daf-cd3e-4403-8b11-9c7e5ce32655 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:28.896023 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1083/2155] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:29.248120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc3215-f7c8-410b-8fa8-3c4ff97ed225 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:29.253675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e1fc3215-f7c8-410b-8fa8-3c4ff97ed225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:29.314827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc3215-f7c8-410b-8fa8-3c4ff97ed225 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:29.318532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba662afa-6ebf-42ab-a3d3-4611950e26e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:29.319167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba662afa-6ebf-42ab-a3d3-4611950e26e8 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:29.319449 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba662afa-6ebf-42ab-a3d3-4611950e26e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:29.319721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba662afa-6ebf-42ab-a3d3-4611950e26e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:29.321534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ba662afa-6ebf-42ab-a3d3-4611950e26e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:29.339985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29771daf-cd3e-4403-8b11-9c7e5ce32655 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:29.341817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-29771daf-cd3e-4403-8b11-9c7e5ce32655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:29.357357 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29771daf-cd3e-4403-8b11-9c7e5ce32655 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:29.362451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:29.363139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:29.363427 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:29.363697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] PATH_INFO: `/v3/projects/a49b52f2a40744eaab8ac62e6bd96807` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:29.364869 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49b52f2a40744eaab8ac62e6bd96807)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:29.366820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:29.393563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba662afa-6ebf-42ab-a3d3-4611950e26e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:29.395013 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba662afa-6ebf-42ab-a3d3-4611950e26e8 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:29.403969 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1073/2156] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:29.476483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49b52f2a40744eaab8ac62e6bd96807', 'request_id': 'req-58f3892e-c5ae-4766-a8cc-6556f854810d'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:29.486260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49b52f2a40744eaab8ac62e6bd96807', 'request_id': 'req-58f3892e-c5ae-4766-a8cc-6556f854810d'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:29.492241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49b52f2a40744eaab8ac62e6bd96807', 'request_id': 'req-58f3892e-c5ae-4766-a8cc-6556f854810d'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:29.605058 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] The token cache is being invalidate because project a49b52f2a40744eaab8ac62e6bd96807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:29.605689 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:29.606158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:29.607927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1084/2157] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:29 2022] DELETE /identity/v3/projects/a49b52f2a40744eaab8ac62e6bd96807 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:29.611869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba662afa-6ebf-42ab-a3d3-4611950e26e8 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:29.623137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba662afa-6ebf-42ab-a3d3-4611950e26e8 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:29.640921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:29.684990 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc86a59-d526-4485-a911-cd7b84809309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:29.685773 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc86a59-d526-4485-a911-cd7b84809309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:29.699512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc86a59-d526-4485-a911-cd7b84809309 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:29.699512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc86a59-d526-4485-a911-cd7b84809309 admin admin] PATH_INFO: `/v3/users/2880d6868dec4335a538705159774349` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:29.699512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc86a59-d526-4485-a911-cd7b84809309 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2880d6868dec4335a538705159774349)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:29.699512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc86a59-d526-4485-a911-cd7b84809309 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:29.904449 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1fc86a59-d526-4485-a911-cd7b84809309 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2880d6868dec4335a538705159774349'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:29.958361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1fc86a59-d526-4485-a911-cd7b84809309 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2880d6868dec4335a538705159774349'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:29.959080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1fc86a59-d526-4485-a911-cd7b84809309 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2880d6868dec4335a538705159774349'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:29.966544 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1fc86a59-d526-4485-a911-cd7b84809309 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2880d6868dec4335a538705159774349'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:29.985498 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1fc86a59-d526-4485-a911-cd7b84809309 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2880d6868dec4335a538705159774349'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:29.993899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1074/2158] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:29 2022] DELETE /identity/v3/users/2880d6868dec4335a538705159774349 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:30.462997 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:30.464532 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:30.478714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3892e-c5ae-4766-a8cc-6556f854810d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:30.490714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:30.548302 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ad173-8706-4e25-ac9d-f175684f0a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:30.549788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ad173-8706-4e25-ac9d-f175684f0a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:30.550075 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ad173-8706-4e25-ac9d-f175684f0a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:30.550352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ad173-8706-4e25-ac9d-f175684f0a4b admin admin] PATH_INFO: `/v3/projects/3b5664710df74519a3db974eddf4e197` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:30.551506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ad173-8706-4e25-ac9d-f175684f0a4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b5664710df74519a3db974eddf4e197)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:30.553515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ad173-8706-4e25-ac9d-f175684f0a4b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:30.690303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-163ad173-8706-4e25-ac9d-f175684f0a4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b5664710df74519a3db974eddf4e197', 'request_id': 'req-163ad173-8706-4e25-ac9d-f175684f0a4b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:30.696393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-163ad173-8706-4e25-ac9d-f175684f0a4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b5664710df74519a3db974eddf4e197', 'request_id': 'req-163ad173-8706-4e25-ac9d-f175684f0a4b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:30.702300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-163ad173-8706-4e25-ac9d-f175684f0a4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b5664710df74519a3db974eddf4e197', 'request_id': 'req-163ad173-8706-4e25-ac9d-f175684f0a4b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:30.816190 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-163ad173-8706-4e25-ac9d-f175684f0a4b admin admin] The token cache is being invalidate because project 3b5664710df74519a3db974eddf4e197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:30.816190 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-163ad173-8706-4e25-ac9d-f175684f0a4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:30.816938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-163ad173-8706-4e25-ac9d-f175684f0a4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:30.818507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1085/2159] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:30 2022] DELETE /identity/v3/projects/3b5664710df74519a3db974eddf4e197 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:30.823475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7020e-afca-465e-b705-6c8ed5a35223 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:30.823765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7020e-afca-465e-b705-6c8ed5a35223 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:30.824047 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7020e-afca-465e-b705-6c8ed5a35223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:30.846345 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-b5c7020e-afca-465e-b705-6c8ed5a35223 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:30.847694 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b5c7020e-afca-465e-b705-6c8ed5a35223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:30.909550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1075/2160] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:53:30.917520 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddcc56-2a10-4db1-a6f4-6df63b39ecbc None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:30.917889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddcc56-2a10-4db1-a6f4-6df63b39ecbc None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:30.918167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddcc56-2a10-4db1-a6f4-6df63b39ecbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:30.955607 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-c4ddcc56-2a10-4db1-a6f4-6df63b39ecbc None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:30.955607 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c4ddcc56-2a10-4db1-a6f4-6df63b39ecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:30.955607 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1086/2161] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:30.956890 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7020e-afca-465e-b705-6c8ed5a35223 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:30.966601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7020e-afca-465e-b705-6c8ed5a35223 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:30.972176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886c196-e787-41fe-8712-113e079adda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:30.972802 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886c196-e787-41fe-8712-113e079adda9 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:30.973166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886c196-e787-41fe-8712-113e079adda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:30.973430 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886c196-e787-41fe-8712-113e079adda9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:30.974329 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886c196-e787-41fe-8712-113e079adda9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:53:30.974612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886c196-e787-41fe-8712-113e079adda9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:30.976193 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886c196-e787-41fe-8712-113e079adda9 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:30.991566 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1076/2162] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:53:30.997676 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddcc56-2a10-4db1-a6f4-6df63b39ecbc None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:31.003365 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddcc56-2a10-4db1-a6f4-6df63b39ecbc None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:31.008016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6e299-db8f-4b53-a297-5dc1e10e73bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:31.008620 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6e299-db8f-4b53-a297-5dc1e10e73bb admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:31.008940 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6e299-db8f-4b53-a297-5dc1e10e73bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:31.009215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6e299-db8f-4b53-a297-5dc1e10e73bb admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:31.010021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6e299-db8f-4b53-a297-5dc1e10e73bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:31.011983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6e299-db8f-4b53-a297-5dc1e10e73bb admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:31.111422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1087/2163] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:30 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:31.115668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886c196-e787-41fe-8712-113e079adda9 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:31.125968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886c196-e787-41fe-8712-113e079adda9 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:31.155039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:31.198701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25ddd0-ad0d-4466-9c2c-63151e69966a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:31.199652 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25ddd0-ad0d-4466-9c2c-63151e69966a admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:31.200105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25ddd0-ad0d-4466-9c2c-63151e69966a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:31.200533 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25ddd0-ad0d-4466-9c2c-63151e69966a admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:31.201704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b25ddd0-ad0d-4466-9c2c-63151e69966a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:31.204221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b25ddd0-ad0d-4466-9c2c-63151e69966a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:31.242811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6e299-db8f-4b53-a297-5dc1e10e73bb admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:31.244632 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-1ff6e299-db8f-4b53-a297-5dc1e10e73bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:31.300637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6e299-db8f-4b53-a297-5dc1e10e73bb admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:31.304305 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678d8fc-e843-4f65-90de-690acc08e8a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:31.304938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678d8fc-e843-4f65-90de-690acc08e8a7 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:31.305220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678d8fc-e843-4f65-90de-690acc08e8a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:31.305494 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678d8fc-e843-4f65-90de-690acc08e8a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:31.307334 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f678d8fc-e843-4f65-90de-690acc08e8a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:31.335171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1077/2164] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:31 2022] POST /identity/v3/users => generated 287 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:31.335661 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:31.336607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25ddd0-ad0d-4466-9c2c-63151e69966a admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:31.346913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25ddd0-ad0d-4466-9c2c-63151e69966a admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:31.351663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2fc8f-3279-4c57-b56b-e398a9d15e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:31.352319 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2fc8f-3279-4c57-b56b-e398a9d15e84 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:31.352610 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2fc8f-3279-4c57-b56b-e398a9d15e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:31.352918 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2fc8f-3279-4c57-b56b-e398a9d15e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:31.373053 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2fc8f-3279-4c57-b56b-e398a9d15e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:31.373053 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2fc8f-3279-4c57-b56b-e398a9d15e84 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:31.373053 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1078/2165] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:53:31.373310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678d8fc-e843-4f65-90de-690acc08e8a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:31.373310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678d8fc-e843-4f65-90de-690acc08e8a7 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:31.376013 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2fc8f-3279-4c57-b56b-e398a9d15e84 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:31.383960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1088/2166] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:53:31.387826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2fc8f-3279-4c57-b56b-e398a9d15e84 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:31.394319 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429aadb-af7b-4aeb-9462-afd6bb17b9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:31.394923 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429aadb-af7b-4aeb-9462-afd6bb17b9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:31.395216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429aadb-af7b-4aeb-9462-afd6bb17b9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:31.395488 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429aadb-af7b-4aeb-9462-afd6bb17b9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:31.396176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429aadb-af7b-4aeb-9462-afd6bb17b9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:31.397929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429aadb-af7b-4aeb-9462-afd6bb17b9c5 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:31.411039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1079/2167] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:31.416389 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678d8fc-e843-4f65-90de-690acc08e8a7 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:31.428925 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678d8fc-e843-4f65-90de-690acc08e8a7 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:31.433799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede5a21-d08f-4fbc-93d0-8530d7212a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:31.434457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede5a21-d08f-4fbc-93d0-8530d7212a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:31.434743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede5a21-d08f-4fbc-93d0-8530d7212a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:31.435021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede5a21-d08f-4fbc-93d0-8530d7212a1b admin admin] PATH_INFO: `/v3/projects/5221c073fdfb4d10adc38d6a18cf7841/users/1951f743398e4b3da175b29819d7c64b/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:31.450306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede5a21-d08f-4fbc-93d0-8530d7212a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5221c073fdfb4d10adc38d6a18cf7841, user_id=1951f743398e4b3da175b29819d7c64b, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:31.452996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede5a21-d08f-4fbc-93d0-8530d7212a1b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:31.554612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1089/2168] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:31 2022] PUT /identity/v3/projects/5221c073fdfb4d10adc38d6a18cf7841/users/1951f743398e4b3da175b29819d7c64b/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:31.559679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ce6b8-d4cb-4630-9f9e-386db2b13bc9 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:31.559964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ce6b8-d4cb-4630-9f9e-386db2b13bc9 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:31.560238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ce6b8-d4cb-4630-9f9e-386db2b13bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:31.587232 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e81ce6b8-d4cb-4630-9f9e-386db2b13bc9 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:31.588493 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e81ce6b8-d4cb-4630-9f9e-386db2b13bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:31.660007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1080/2169] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:53:31.669964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede5a21-d08f-4fbc-93d0-8530d7212a1b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:31.671723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2ede5a21-d08f-4fbc-93d0-8530d7212a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:31.730563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede5a21-d08f-4fbc-93d0-8530d7212a1b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:31.734242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ff6e9-c20b-4c33-8476-85e350f072cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:31.734839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ff6e9-c20b-4c33-8476-85e350f072cb service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:31.735119 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ff6e9-c20b-4c33-8476-85e350f072cb service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:31.735392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ff6e9-c20b-4c33-8476-85e350f072cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:31.744958 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ff6e9-c20b-4c33-8476-85e350f072cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:31.757259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ff6e9-c20b-4c33-8476-85e350f072cb service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:31.757259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1090/2170] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:32.273924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ce6b8-d4cb-4630-9f9e-386db2b13bc9 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:32.275349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e81ce6b8-d4cb-4630-9f9e-386db2b13bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:32.288342 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ce6b8-d4cb-4630-9f9e-386db2b13bc9 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:32.294335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a2e1f-27c0-4312-a743-67957d191f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:32.295048 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a2e1f-27c0-4312-a743-67957d191f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:32.295326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a2e1f-27c0-4312-a743-67957d191f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:32.295598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a2e1f-27c0-4312-a743-67957d191f2e admin admin] PATH_INFO: `/v3/users/84db43dae1874ee4b7d2f7741bbfc360` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:32.296786 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a2e1f-27c0-4312-a743-67957d191f2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=84db43dae1874ee4b7d2f7741bbfc360)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:32.299227 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a2e1f-27c0-4312-a743-67957d191f2e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:32.471797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fe3a2e1f-27c0-4312-a743-67957d191f2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84db43dae1874ee4b7d2f7741bbfc360'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:32.542357 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fe3a2e1f-27c0-4312-a743-67957d191f2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84db43dae1874ee4b7d2f7741bbfc360'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:32.543007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fe3a2e1f-27c0-4312-a743-67957d191f2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84db43dae1874ee4b7d2f7741bbfc360'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:32.551199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fe3a2e1f-27c0-4312-a743-67957d191f2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84db43dae1874ee4b7d2f7741bbfc360'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:32.563261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fe3a2e1f-27c0-4312-a743-67957d191f2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84db43dae1874ee4b7d2f7741bbfc360'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:32.569786 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1081/2171] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:32 2022] DELETE /identity/v3/users/84db43dae1874ee4b7d2f7741bbfc360 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:32.581332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6f093-d73a-44d4-9f76-3a5a3416c7ae None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:32.582164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6f093-d73a-44d4-9f76-3a5a3416c7ae None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:32.582164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6f093-d73a-44d4-9f76-3a5a3416c7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:32.602063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-b0d6f093-d73a-44d4-9f76-3a5a3416c7ae None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:32.603274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b0d6f093-d73a-44d4-9f76-3a5a3416c7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:32.672779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1091/2172] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:32.683784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a2e1f-27c0-4312-a743-67957d191f2e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:32.685303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fe3a2e1f-27c0-4312-a743-67957d191f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:32.697301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:32.749645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a2e1f-27c0-4312-a743-67957d191f2e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:32.754286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c8c4b-f4ff-46b9-aae0-88a14e07fe68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:32.754976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c8c4b-f4ff-46b9-aae0-88a14e07fe68 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:32.755279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c8c4b-f4ff-46b9-aae0-88a14e07fe68 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:32.755573 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c8c4b-f4ff-46b9-aae0-88a14e07fe68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:32.768013 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c8c4b-f4ff-46b9-aae0-88a14e07fe68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:32.768013 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c8c4b-f4ff-46b9-aae0-88a14e07fe68 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:32.779584 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1082/2173] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:33.235538 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c8c4b-f4ff-46b9-aae0-88a14e07fe68 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:33.237222 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b4b0c-996a-4298-9047-22b39aee4e2f None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:33.238370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b4b0c-996a-4298-9047-22b39aee4e2f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:33.238806 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b4b0c-996a-4298-9047-22b39aee4e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:33.239141 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d22c8c4b-f4ff-46b9-aae0-88a14e07fe68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:33.289238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-e36b4b0c-996a-4298-9047-22b39aee4e2f None None] MFA Rules not processed for user `1951f743398e4b3da175b29819d7c64b`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:33.290394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e36b4b0c-996a-4298-9047-22b39aee4e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:33.299080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c8c4b-f4ff-46b9-aae0-88a14e07fe68 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:33.305288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1c06c-7688-430f-ab37-2b86e5dd074b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:33.305919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1c06c-7688-430f-ab37-2b86e5dd074b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:33.306348 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1c06c-7688-430f-ab37-2b86e5dd074b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:33.306766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1c06c-7688-430f-ab37-2b86e5dd074b admin admin] PATH_INFO: `/v3/projects/bb370e5e106a4f73b4f9602e132d84d3` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:33.308061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1c06c-7688-430f-ab37-2b86e5dd074b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb370e5e106a4f73b4f9602e132d84d3)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:33.310347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1c06c-7688-430f-ab37-2b86e5dd074b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:33.340671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:33.376151 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1092/2174] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:33.383088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b4b0c-996a-4298-9047-22b39aee4e2f None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:33.383088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e36b4b0c-996a-4298-9047-22b39aee4e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:33.400597 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b4b0c-996a-4298-9047-22b39aee4e2f None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:33.426629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc574f7-cab5-4caf-80b3-49a7b8b03118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:33.427054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc574f7-cab5-4caf-80b3-49a7b8b03118 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:33.427350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc574f7-cab5-4caf-80b3-49a7b8b03118 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:33.427634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc574f7-cab5-4caf-80b3-49a7b8b03118 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:33.428273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc574f7-cab5-4caf-80b3-49a7b8b03118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:33.429277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e5b1c06c-7688-430f-ab37-2b86e5dd074b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb370e5e106a4f73b4f9602e132d84d3', 'request_id': 'req-e5b1c06c-7688-430f-ab37-2b86e5dd074b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:33.430230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc574f7-cab5-4caf-80b3-49a7b8b03118 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:33.438224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e5b1c06c-7688-430f-ab37-2b86e5dd074b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb370e5e106a4f73b4f9602e132d84d3', 'request_id': 'req-e5b1c06c-7688-430f-ab37-2b86e5dd074b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:33.524385 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e5b1c06c-7688-430f-ab37-2b86e5dd074b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb370e5e106a4f73b4f9602e132d84d3', 'request_id': 'req-e5b1c06c-7688-430f-ab37-2b86e5dd074b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:33.536165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e5b1c06c-7688-430f-ab37-2b86e5dd074b admin admin] The token cache is being invalidate because project bb370e5e106a4f73b4f9602e132d84d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:33.536486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e5b1c06c-7688-430f-ab37-2b86e5dd074b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:33.537158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e5b1c06c-7688-430f-ab37-2b86e5dd074b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:33.538674 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1083/2175] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:33 2022] DELETE /identity/v3/projects/bb370e5e106a4f73b4f9602e132d84d3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:33.543402 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4b904-e1e5-4fb1-a785-1b714428c470 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:33.543730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4b904-e1e5-4fb1-a785-1b714428c470 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:33.544080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4b904-e1e5-4fb1-a785-1b714428c470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:33.555704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1093/2176] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:33 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:53:33.556573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc574f7-cab5-4caf-80b3-49a7b8b03118 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:33.558281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-edc574f7-cab5-4caf-80b3-49a7b8b03118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:33.567380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-74b4b904-e1e5-4fb1-a785-1b714428c470 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:33.568583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-74b4b904-e1e5-4fb1-a785-1b714428c470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:33.633413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc574f7-cab5-4caf-80b3-49a7b8b03118 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:33.636339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1084/2177] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:53:33.639658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f0395-2a03-470e-9834-a88bcbf0263d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:33.640493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f0395-2a03-470e-9834-a88bcbf0263d admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:33.640945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f0395-2a03-470e-9834-a88bcbf0263d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:33.641383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f0395-2a03-470e-9834-a88bcbf0263d admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:33.642218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f0395-2a03-470e-9834-a88bcbf0263d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:33.644082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7bd08-e423-4183-aa0f-131ae00b213f None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:33.644378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7bd08-e423-4183-aa0f-131ae00b213f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:33.644685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7bd08-e423-4183-aa0f-131ae00b213f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:33.646018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f0395-2a03-470e-9834-a88bcbf0263d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:33.665570 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-98b7bd08-e423-4183-aa0f-131ae00b213f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:33.666803 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-98b7bd08-e423-4183-aa0f-131ae00b213f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:33.676525 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1085/2178] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:53:33.681297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7bd08-e423-4183-aa0f-131ae00b213f None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:33.692237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7bd08-e423-4183-aa0f-131ae00b213f None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:33.698177 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf93963-d6ca-4b99-bdcc-46b0d0b207d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:33.698817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf93963-d6ca-4b99-bdcc-46b0d0b207d2 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:33.699097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf93963-d6ca-4b99-bdcc-46b0d0b207d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:33.699982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf93963-d6ca-4b99-bdcc-46b0d0b207d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:33.700092 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf93963-d6ca-4b99-bdcc-46b0d0b207d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:53:33.700371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf93963-d6ca-4b99-bdcc-46b0d0b207d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:33.702007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf93963-d6ca-4b99-bdcc-46b0d0b207d2 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:33.722526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1086/2179] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:53:33.722981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1094/2180] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:33 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:33.726654 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf93963-d6ca-4b99-bdcc-46b0d0b207d2 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:33.727573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f0395-2a03-470e-9834-a88bcbf0263d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:33.736633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf93963-d6ca-4b99-bdcc-46b0d0b207d2 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:33.738235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f0395-2a03-470e-9834-a88bcbf0263d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:33.741302 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4dd9c-a1c2-4524-b5db-68365b08019a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:33.741920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4dd9c-a1c2-4524-b5db-68365b08019a admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:33.743662 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4dd9c-a1c2-4524-b5db-68365b08019a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:33.743662 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4dd9c-a1c2-4524-b5db-68365b08019a admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:33.743856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b616010-6fd5-4ddd-a231-f1f29a99820d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:33.743961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4dd9c-a1c2-4524-b5db-68365b08019a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:33.744039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616010-6fd5-4ddd-a231-f1f29a99820d admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:33.744039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616010-6fd5-4ddd-a231-f1f29a99820d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:33.744279 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616010-6fd5-4ddd-a231-f1f29a99820d admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:33.745071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b616010-6fd5-4ddd-a231-f1f29a99820d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:33.745220 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4dd9c-a1c2-4524-b5db-68365b08019a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:33.746656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b616010-6fd5-4ddd-a231-f1f29a99820d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:33.757164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1095/2181] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:33.761272 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b616010-6fd5-4ddd-a231-f1f29a99820d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:33.771833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b616010-6fd5-4ddd-a231-f1f29a99820d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:33.776427 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168c739-2934-424e-b5bc-12e2c407730f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:33.777101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168c739-2934-424e-b5bc-12e2c407730f admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:33.777386 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168c739-2934-424e-b5bc-12e2c407730f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:33.777662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168c739-2934-424e-b5bc-12e2c407730f admin admin] PATH_INFO: `/v3/projects/fd8b2b88b1cf4ee6b05596ae9f30431d/users/0f668737502f4e8fb7c97c0cadc1befa/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:33.801015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168c739-2934-424e-b5bc-12e2c407730f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd8b2b88b1cf4ee6b05596ae9f30431d, user_id=0f668737502f4e8fb7c97c0cadc1befa, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:33.801015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168c739-2934-424e-b5bc-12e2c407730f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:33.860280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1087/2182] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:33 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:53:33.862392 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4dd9c-a1c2-4524-b5db-68365b08019a admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:33.902307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1096/2183] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:33 2022] PUT /identity/v3/projects/fd8b2b88b1cf4ee6b05596ae9f30431d/users/0f668737502f4e8fb7c97c0cadc1befa/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:33.906316 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168c739-2934-424e-b5bc-12e2c407730f admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:33.925339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4dd9c-a1c2-4524-b5db-68365b08019a admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:33.968957 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168c739-2934-424e-b5bc-12e2c407730f admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:33.974496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005e8b7-f934-4c31-b647-f5e8df7c757c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:33.975139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005e8b7-f934-4c31-b647-f5e8df7c757c admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:33.975420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005e8b7-f934-4c31-b647-f5e8df7c757c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:33.975690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005e8b7-f934-4c31-b647-f5e8df7c757c admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:33.976335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005e8b7-f934-4c31-b647-f5e8df7c757c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:33.977967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005e8b7-f934-4c31-b647-f5e8df7c757c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:33.983491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3c0f9-167d-4e96-afe1-a774c8f89ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:33.984098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3c0f9-167d-4e96-afe1-a774c8f89ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:33.984378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3c0f9-167d-4e96-afe1-a774c8f89ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:33.984751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3c0f9-167d-4e96-afe1-a774c8f89ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:33.985373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3c0f9-167d-4e96-afe1-a774c8f89ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:33.987849 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3c0f9-167d-4e96-afe1-a774c8f89ca9 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:33.988352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1097/2184] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:53:33.992383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005e8b7-f934-4c31-b647-f5e8df7c757c admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:34.002397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005e8b7-f934-4c31-b647-f5e8df7c757c admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:34.007098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816d871-f1eb-48e3-9438-ea3a0f57c597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:34.007737 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816d871-f1eb-48e3-9438-ea3a0f57c597 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:34.008057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816d871-f1eb-48e3-9438-ea3a0f57c597 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:34.008337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816d871-f1eb-48e3-9438-ea3a0f57c597 admin admin] PATH_INFO: `/v3/projects/fd8b2b88b1cf4ee6b05596ae9f30431d/users/0f668737502f4e8fb7c97c0cadc1befa/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:34.010355 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816d871-f1eb-48e3-9438-ea3a0f57c597 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd8b2b88b1cf4ee6b05596ae9f30431d, user_id=0f668737502f4e8fb7c97c0cadc1befa, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:34.013018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816d871-f1eb-48e3-9438-ea3a0f57c597 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:34.023611 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1098/2185] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:33 2022] PUT /identity/v3/projects/fd8b2b88b1cf4ee6b05596ae9f30431d/users/0f668737502f4e8fb7c97c0cadc1befa/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:34.068473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1088/2186] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:33 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:53:34.072709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816d871-f1eb-48e3-9438-ea3a0f57c597 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:34.135390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816d871-f1eb-48e3-9438-ea3a0f57c597 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:34.140078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf38b2-a3aa-4719-b7c6-9bcf55fbc027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:34.140749 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf38b2-a3aa-4719-b7c6-9bcf55fbc027 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:34.141028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf38b2-a3aa-4719-b7c6-9bcf55fbc027 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:34.141297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf38b2-a3aa-4719-b7c6-9bcf55fbc027 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:34.141960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf38b2-a3aa-4719-b7c6-9bcf55fbc027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:34.144126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf38b2-a3aa-4719-b7c6-9bcf55fbc027 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:34.154949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1099/2187] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:34.159233 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3c0f9-167d-4e96-afe1-a774c8f89ca9 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:34.169372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3c0f9-167d-4e96-afe1-a774c8f89ca9 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:34.171792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:34.176126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e6f45-da56-48e5-bc3b-f2ee6d81b1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:34.176810 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e6f45-da56-48e5-bc3b-f2ee6d81b1ed admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:34.177141 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e6f45-da56-48e5-bc3b-f2ee6d81b1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:34.177368 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e6f45-da56-48e5-bc3b-f2ee6d81b1ed admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:34.178043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e6f45-da56-48e5-bc3b-f2ee6d81b1ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:34.179640 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e6f45-da56-48e5-bc3b-f2ee6d81b1ed admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:34.191253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1089/2188] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:34.195530 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf38b2-a3aa-4719-b7c6-9bcf55fbc027 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:34.206509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf38b2-a3aa-4719-b7c6-9bcf55fbc027 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:34.211282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d7ee8-ae08-41d5-bec6-a32d79bbc0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:34.211945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d7ee8-ae08-41d5-bec6-a32d79bbc0ba admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:34.212232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d7ee8-ae08-41d5-bec6-a32d79bbc0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:34.212527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d7ee8-ae08-41d5-bec6-a32d79bbc0ba admin admin] PATH_INFO: `/v3/projects/c075cdffd0844d5fa300444bf0727ce5/users/ac056c41d17d48c4a82e9d1ce02127db/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:34.227118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d7ee8-ae08-41d5-bec6-a32d79bbc0ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=c075cdffd0844d5fa300444bf0727ce5, user_id=ac056c41d17d48c4a82e9d1ce02127db, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:34.229811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d7ee8-ae08-41d5-bec6-a32d79bbc0ba admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:34.315279 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:34.325319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1100/2189] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:34 2022] PUT /identity/v3/projects/c075cdffd0844d5fa300444bf0727ce5/users/ac056c41d17d48c4a82e9d1ce02127db/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:34.338103 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836760e-f731-4615-b80a-54e13869c0c7 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:34.338529 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836760e-f731-4615-b80a-54e13869c0c7 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:34.338529 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836760e-f731-4615-b80a-54e13869c0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:34.391123 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d836760e-f731-4615-b80a-54e13869c0c7 None None] MFA Rules not processed for user `ac056c41d17d48c4a82e9d1ce02127db`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:34.393180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d836760e-f731-4615-b80a-54e13869c0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:34.467290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1090/2190] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:34.487303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d7ee8-ae08-41d5-bec6-a32d79bbc0ba admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:34.489221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6f6d7ee8-ae08-41d5-bec6-a32d79bbc0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:34.569164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d7ee8-ae08-41d5-bec6-a32d79bbc0ba admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:34.573321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d13d9-7746-44c7-beb3-405718752152 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:34.573874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d13d9-7746-44c7-beb3-405718752152 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:34.574158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d13d9-7746-44c7-beb3-405718752152 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:34.574435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d13d9-7746-44c7-beb3-405718752152 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:34.583630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d13d9-7746-44c7-beb3-405718752152 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:34.585118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d13d9-7746-44c7-beb3-405718752152 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:34.595279 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1101/2191] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:34.641362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836760e-f731-4615-b80a-54e13869c0c7 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:34.748712 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836760e-f731-4615-b80a-54e13869c0c7 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:34.755078 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d82f0-5d40-4b1b-988b-c9d855069026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:34.755838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d82f0-5d40-4b1b-988b-c9d855069026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:34.756310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d82f0-5d40-4b1b-988b-c9d855069026 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:34.756711 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d82f0-5d40-4b1b-988b-c9d855069026 admin admin] PATH_INFO: `/v3/users/ac056c41d17d48c4a82e9d1ce02127db` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:34.758121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d82f0-5d40-4b1b-988b-c9d855069026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac056c41d17d48c4a82e9d1ce02127db)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:34.760132 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d82f0-5d40-4b1b-988b-c9d855069026 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:34.989187 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5b3d82f0-5d40-4b1b-988b-c9d855069026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac056c41d17d48c4a82e9d1ce02127db'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:35.066823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5b3d82f0-5d40-4b1b-988b-c9d855069026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac056c41d17d48c4a82e9d1ce02127db'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:35.067445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5b3d82f0-5d40-4b1b-988b-c9d855069026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac056c41d17d48c4a82e9d1ce02127db'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:35.074853 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5b3d82f0-5d40-4b1b-988b-c9d855069026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac056c41d17d48c4a82e9d1ce02127db'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:35.087226 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5b3d82f0-5d40-4b1b-988b-c9d855069026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac056c41d17d48c4a82e9d1ce02127db'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:35.093183 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1091/2192] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:34 2022] DELETE /identity/v3/users/ac056c41d17d48c4a82e9d1ce02127db => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:35.098445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f83958-7dbe-47b7-b168-67d7d600f11c None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:35.098736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f83958-7dbe-47b7-b168-67d7d600f11c None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:35.099021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f83958-7dbe-47b7-b168-67d7d600f11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:35.118922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-57f83958-7dbe-47b7-b168-67d7d600f11c None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:35.120046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-57f83958-7dbe-47b7-b168-67d7d600f11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:35.208725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1102/2193] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:35.218542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d82f0-5d40-4b1b-988b-c9d855069026 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:35.220998 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5b3d82f0-5d40-4b1b-988b-c9d855069026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:35.317112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d82f0-5d40-4b1b-988b-c9d855069026 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:35.320832 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f15a98-10e0-40c2-b69c-b56547adf802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:35.321435 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f15a98-10e0-40c2-b69c-b56547adf802 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:35.321719 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f15a98-10e0-40c2-b69c-b56547adf802 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:35.321995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f15a98-10e0-40c2-b69c-b56547adf802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:35.333468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f15a98-10e0-40c2-b69c-b56547adf802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:35.334931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f15a98-10e0-40c2-b69c-b56547adf802 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:35.345090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1092/2194] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:35.515081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35f7d9-3839-4aee-8bec-8e6a46d3b0ee None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:35.515646 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35f7d9-3839-4aee-8bec-8e6a46d3b0ee None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:35.515646 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35f7d9-3839-4aee-8bec-8e6a46d3b0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:35.537155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:35.565607 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-4b35f7d9-3839-4aee-8bec-8e6a46d3b0ee None None] MFA Rules not processed for user `0f668737502f4e8fb7c97c0cadc1befa`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:35.593006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4b35f7d9-3839-4aee-8bec-8e6a46d3b0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:35.708694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1103/2195] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:35.718999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f15a98-10e0-40c2-b69c-b56547adf802 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:35.724031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-45f15a98-10e0-40c2-b69c-b56547adf802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:35.788074 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f15a98-10e0-40c2-b69c-b56547adf802 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:35.791686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d941203-958c-44b9-92d9-127af426c226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:35.792308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d941203-958c-44b9-92d9-127af426c226 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:35.792589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d941203-958c-44b9-92d9-127af426c226 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:35.792908 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d941203-958c-44b9-92d9-127af426c226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:35.794627 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0d941203-958c-44b9-92d9-127af426c226 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:35.857026 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d941203-958c-44b9-92d9-127af426c226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:35.858552 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d941203-958c-44b9-92d9-127af426c226 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:35.869640 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1093/2196] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:35.890221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35f7d9-3839-4aee-8bec-8e6a46d3b0ee None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:35.891949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4b35f7d9-3839-4aee-8bec-8e6a46d3b0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:35.917880 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35f7d9-3839-4aee-8bec-8e6a46d3b0ee None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:35.926719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:35.928433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:35.929012 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:35.929666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b admin admin] PATH_INFO: `/v3/projects/c075cdffd0844d5fa300444bf0727ce5` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:35.931641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c075cdffd0844d5fa300444bf0727ce5)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:35.933935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:36.080919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c075cdffd0844d5fa300444bf0727ce5', 'request_id': 'req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:36.087290 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c075cdffd0844d5fa300444bf0727ce5', 'request_id': 'req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:36.093380 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c075cdffd0844d5fa300444bf0727ce5', 'request_id': 'req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:36.153681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:36.158365 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b admin admin] The token cache is being invalidate because project c075cdffd0844d5fa300444bf0727ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:36.158668 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:36.159319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-9d4c2eb7-aa2f-445e-905e-00c8ccaa3c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:36.160871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1104/2197] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:35 2022] DELETE /identity/v3/projects/c075cdffd0844d5fa300444bf0727ce5 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:36.165874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436261c6-9321-4798-a9ac-8c95989121ff None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:36.166161 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436261c6-9321-4798-a9ac-8c95989121ff None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:36.166439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436261c6-9321-4798-a9ac-8c95989121ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:36.185763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-436261c6-9321-4798-a9ac-8c95989121ff None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:36.186948 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-436261c6-9321-4798-a9ac-8c95989121ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:36.249152 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1094/2198] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:36.257587 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66257ee2-8005-412a-a4ae-5eb7e9901ed0 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:36.257880 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66257ee2-8005-412a-a4ae-5eb7e9901ed0 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:36.258159 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66257ee2-8005-412a-a4ae-5eb7e9901ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:36.278254 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-66257ee2-8005-412a-a4ae-5eb7e9901ed0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:36.279485 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-66257ee2-8005-412a-a4ae-5eb7e9901ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:36.288414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1105/2199] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:36.292879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436261c6-9321-4798-a9ac-8c95989121ff None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:36.303882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436261c6-9321-4798-a9ac-8c95989121ff None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:36.309019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289eeea-3d59-4572-b17c-c1d62c88588b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:36.309662 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289eeea-3d59-4572-b17c-c1d62c88588b admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:36.309946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289eeea-3d59-4572-b17c-c1d62c88588b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:36.310251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289eeea-3d59-4572-b17c-c1d62c88588b admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:36.310905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289eeea-3d59-4572-b17c-c1d62c88588b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:53:36.311216 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289eeea-3d59-4572-b17c-c1d62c88588b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:36.312923 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289eeea-3d59-4572-b17c-c1d62c88588b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:36.326722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1095/2200] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:53:36.330951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66257ee2-8005-412a-a4ae-5eb7e9901ed0 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:36.340307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66257ee2-8005-412a-a4ae-5eb7e9901ed0 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:36.345089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb99655-e865-4cb0-8edf-30f6ca3b42b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:36.349484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb99655-e865-4cb0-8edf-30f6ca3b42b7 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:36.349484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb99655-e865-4cb0-8edf-30f6ca3b42b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:36.349484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb99655-e865-4cb0-8edf-30f6ca3b42b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:36.349484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb99655-e865-4cb0-8edf-30f6ca3b42b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:36.349484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb99655-e865-4cb0-8edf-30f6ca3b42b7 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:36.451714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1106/2201] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:36 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:36.456512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289eeea-3d59-4572-b17c-c1d62c88588b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:36.466733 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289eeea-3d59-4572-b17c-c1d62c88588b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:36.529581 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45e1bb-a1a6-4cb8-86b0-bdff6e4ca77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:36.530200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45e1bb-a1a6-4cb8-86b0-bdff6e4ca77a admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:36.530480 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45e1bb-a1a6-4cb8-86b0-bdff6e4ca77a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:36.530753 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45e1bb-a1a6-4cb8-86b0-bdff6e4ca77a admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:36.531396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45e1bb-a1a6-4cb8-86b0-bdff6e4ca77a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:36.534140 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45e1bb-a1a6-4cb8-86b0-bdff6e4ca77a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:36.635318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1096/2202] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:36 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:36.635819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb99655-e865-4cb0-8edf-30f6ca3b42b7 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:36.648182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb99655-e865-4cb0-8edf-30f6ca3b42b7 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:36.653524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e67aa-1457-43e9-8f30-0dbbe1be839a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:36.654360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e67aa-1457-43e9-8f30-0dbbe1be839a admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:36.654756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e67aa-1457-43e9-8f30-0dbbe1be839a admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:36.655015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e67aa-1457-43e9-8f30-0dbbe1be839a admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:36.655948 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e67aa-1457-43e9-8f30-0dbbe1be839a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:36.658082 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e67aa-1457-43e9-8f30-0dbbe1be839a admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:36.670720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1107/2203] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:36.675785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45e1bb-a1a6-4cb8-86b0-bdff6e4ca77a admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:36.691867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e67aa-1457-43e9-8f30-0dbbe1be839a admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:36.692120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45e1bb-a1a6-4cb8-86b0-bdff6e4ca77a admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:36.692216 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-368e67aa-1457-43e9-8f30-0dbbe1be839a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:36.696736 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120a409-ca8b-4ac0-8faa-c640cab89fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:36.700595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120a409-ca8b-4ac0-8faa-c640cab89fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:36.700595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120a409-ca8b-4ac0-8faa-c640cab89fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:36.700595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120a409-ca8b-4ac0-8faa-c640cab89fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:36.700595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120a409-ca8b-4ac0-8faa-c640cab89fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:36.700595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120a409-ca8b-4ac0-8faa-c640cab89fd2 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:36.704114 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e67aa-1457-43e9-8f30-0dbbe1be839a admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:36.709720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cecd5-c4e1-4ae1-a05d-048bbc17107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:36.710344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cecd5-c4e1-4ae1-a05d-048bbc17107c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:36.710624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cecd5-c4e1-4ae1-a05d-048bbc17107c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:36.710895 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cecd5-c4e1-4ae1-a05d-048bbc17107c admin admin] PATH_INFO: `/v3/users/a238c069d143404fa73cf1f955fba883` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:36.712031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cecd5-c4e1-4ae1-a05d-048bbc17107c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a238c069d143404fa73cf1f955fba883)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:36.712785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1097/2204] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:36.714064 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cecd5-c4e1-4ae1-a05d-048bbc17107c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:36.717792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120a409-ca8b-4ac0-8faa-c640cab89fd2 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:36.728715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120a409-ca8b-4ac0-8faa-c640cab89fd2 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:36.734628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191e3b5-b14a-4095-99ff-5fce5cfb0442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:36.735746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191e3b5-b14a-4095-99ff-5fce5cfb0442 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:36.736411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191e3b5-b14a-4095-99ff-5fce5cfb0442 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:36.736733 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191e3b5-b14a-4095-99ff-5fce5cfb0442 admin admin] PATH_INFO: `/v3/projects/7d8b8ee42f4d499499f24e29d7a8249d/users/7f3a2203128f46528a40a9f55b54b5b9/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:36.752559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191e3b5-b14a-4095-99ff-5fce5cfb0442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d8b8ee42f4d499499f24e29d7a8249d, user_id=7f3a2203128f46528a40a9f55b54b5b9, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:36.755656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191e3b5-b14a-4095-99ff-5fce5cfb0442 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:36.919830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1098/2205] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:36 2022] PUT /identity/v3/projects/7d8b8ee42f4d499499f24e29d7a8249d/users/7f3a2203128f46528a40a9f55b54b5b9/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:36.925071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15275440-8566-4d05-a1e4-95e46eab7d76 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:36.925510 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15275440-8566-4d05-a1e4-95e46eab7d76 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:36.925790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15275440-8566-4d05-a1e4-95e46eab7d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:36.928821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-998cecd5-c4e1-4ae1-a05d-048bbc17107c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a238c069d143404fa73cf1f955fba883'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:36.937985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-998cecd5-c4e1-4ae1-a05d-048bbc17107c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a238c069d143404fa73cf1f955fba883'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:36.949792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-15275440-8566-4d05-a1e4-95e46eab7d76 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:36.949792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-15275440-8566-4d05-a1e4-95e46eab7d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:37.008526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1099/2206] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:53:37.018838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15275440-8566-4d05-a1e4-95e46eab7d76 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:37.020528 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-15275440-8566-4d05-a1e4-95e46eab7d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:37.076801 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15275440-8566-4d05-a1e4-95e46eab7d76 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:37.079386 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-998cecd5-c4e1-4ae1-a05d-048bbc17107c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a238c069d143404fa73cf1f955fba883'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:37.080762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-998cecd5-c4e1-4ae1-a05d-048bbc17107c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a238c069d143404fa73cf1f955fba883'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:37.081730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4896af-798c-43aa-b37a-2db408a3a95e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:37.082344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4896af-798c-43aa-b37a-2db408a3a95e service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:37.082619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4896af-798c-43aa-b37a-2db408a3a95e service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:37.082898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4896af-798c-43aa-b37a-2db408a3a95e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:37.092128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4896af-798c-43aa-b37a-2db408a3a95e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:37.093598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4896af-798c-43aa-b37a-2db408a3a95e service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:37.095640 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-998cecd5-c4e1-4ae1-a05d-048bbc17107c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a238c069d143404fa73cf1f955fba883'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:37.101590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1108/2207] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:36 2022] DELETE /identity/v3/users/a238c069d143404fa73cf1f955fba883 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:37.103390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1100/2208] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:53:37.105983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8feebb6-08c4-41e4-8c37-091f92fc698c None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:37.106271 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8feebb6-08c4-41e4-8c37-091f92fc698c None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:37.106548 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8feebb6-08c4-41e4-8c37-091f92fc698c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:37.126038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a8feebb6-08c4-41e4-8c37-091f92fc698c None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:37.128134 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a8feebb6-08c4-41e4-8c37-091f92fc698c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:37.156726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:37.192503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1109/2209] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:53:37.202252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4896af-798c-43aa-b37a-2db408a3a95e service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:37.211769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4896af-798c-43aa-b37a-2db408a3a95e service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:37.263390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bcd71-ff55-4afa-bd4f-56abbf0e38b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:37.263914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bcd71-ff55-4afa-bd4f-56abbf0e38b8 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:37.264192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bcd71-ff55-4afa-bd4f-56abbf0e38b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:37.264462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bcd71-ff55-4afa-bd4f-56abbf0e38b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:37.274769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495bcd71-ff55-4afa-bd4f-56abbf0e38b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:37.276207 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495bcd71-ff55-4afa-bd4f-56abbf0e38b8 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:37.286709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1101/2210] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:37.739927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8feebb6-08c4-41e4-8c37-091f92fc698c None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:37.739927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a8feebb6-08c4-41e4-8c37-091f92fc698c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:37.802161 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bcd71-ff55-4afa-bd4f-56abbf0e38b8 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:37.804379 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-495bcd71-ff55-4afa-bd4f-56abbf0e38b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:37.820255 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bcd71-ff55-4afa-bd4f-56abbf0e38b8 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:37.825583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:37.826181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:37.826458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:37.826901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] PATH_INFO: `/v3/projects/b3bf9d51878b413aac99dbcf7f969643` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:37.828199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3bf9d51878b413aac99dbcf7f969643)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:37.831260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:37.856184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8feebb6-08c4-41e4-8c37-091f92fc698c None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:37.860854 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ace2d-f596-4582-9866-1f2cc30109b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:37.861580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ace2d-f596-4582-9866-1f2cc30109b1 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:37.861865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ace2d-f596-4582-9866-1f2cc30109b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:37.862140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ace2d-f596-4582-9866-1f2cc30109b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:37.863890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-dd1ace2d-f596-4582-9866-1f2cc30109b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:37.925496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ace2d-f596-4582-9866-1f2cc30109b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:37.926944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ace2d-f596-4582-9866-1f2cc30109b1 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:37.936374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1110/2211] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:37.968324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:37.990299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3bf9d51878b413aac99dbcf7f969643', 'request_id': 'req-fd367379-a9c4-45e8-8e00-015219b684ce'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:37.997460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3bf9d51878b413aac99dbcf7f969643', 'request_id': 'req-fd367379-a9c4-45e8-8e00-015219b684ce'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:38.072735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3bf9d51878b413aac99dbcf7f969643', 'request_id': 'req-fd367379-a9c4-45e8-8e00-015219b684ce'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:38.086625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] The token cache is being invalidate because project b3bf9d51878b413aac99dbcf7f969643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:38.086745 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:38.087647 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:38.089880 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1102/2212] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:37 2022] DELETE /identity/v3/projects/b3bf9d51878b413aac99dbcf7f969643 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:38.093986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ace2d-f596-4582-9866-1f2cc30109b1 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:38.108604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ace2d-f596-4582-9866-1f2cc30109b1 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:38.202190 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec7c14-57c2-4422-afd8-25530ceedc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:38.203427 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec7c14-57c2-4422-afd8-25530ceedc83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:38.204132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec7c14-57c2-4422-afd8-25530ceedc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:38.204587 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec7c14-57c2-4422-afd8-25530ceedc83 admin admin] PATH_INFO: `/v3/users/8134ef40af76455d8d7a334957fc1f83` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:38.206288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec7c14-57c2-4422-afd8-25530ceedc83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8134ef40af76455d8d7a334957fc1f83)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:38.208433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec7c14-57c2-4422-afd8-25530ceedc83 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:38.485314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7bec7c14-57c2-4422-afd8-25530ceedc83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8134ef40af76455d8d7a334957fc1f83'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:38.498411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7bec7c14-57c2-4422-afd8-25530ceedc83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8134ef40af76455d8d7a334957fc1f83'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:38.615954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7bec7c14-57c2-4422-afd8-25530ceedc83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8134ef40af76455d8d7a334957fc1f83'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:38.616699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7bec7c14-57c2-4422-afd8-25530ceedc83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8134ef40af76455d8d7a334957fc1f83'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:38.631252 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7bec7c14-57c2-4422-afd8-25530ceedc83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8134ef40af76455d8d7a334957fc1f83'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:38.638598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1111/2213] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:38 2022] DELETE /identity/v3/users/8134ef40af76455d8d7a334957fc1f83 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:39.223961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:39.225149 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:39.239351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd367379-a9c4-45e8-8e00-015219b684ce admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:39.310375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:39.311196 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:39.311684 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:39.312089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] PATH_INFO: `/v3/projects/f28c9de2c1e5451aa1ad76d3b18be5bb` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:39.313618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f28c9de2c1e5451aa1ad76d3b18be5bb)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:39.316797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:39.439596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:39.465219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f28c9de2c1e5451aa1ad76d3b18be5bb', 'request_id': 'req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:39.477961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f28c9de2c1e5451aa1ad76d3b18be5bb', 'request_id': 'req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:39.553497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f28c9de2c1e5451aa1ad76d3b18be5bb', 'request_id': 'req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:39.568457 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] The token cache is being invalidate because project f28c9de2c1e5451aa1ad76d3b18be5bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:39.569381 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:39.573275 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:39.575289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1103/2214] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:39 2022] DELETE /identity/v3/projects/f28c9de2c1e5451aa1ad76d3b18be5bb => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:39.579566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec7c14-57c2-4422-afd8-25530ceedc83 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:39.581238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7bec7c14-57c2-4422-afd8-25530ceedc83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:39.597785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec7c14-57c2-4422-afd8-25530ceedc83 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:39.732868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9040a-5d2b-4fbe-a774-34240053e067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:39.733522 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9040a-5d2b-4fbe-a774-34240053e067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:39.733805 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9040a-5d2b-4fbe-a774-34240053e067 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:39.734120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9040a-5d2b-4fbe-a774-34240053e067 admin admin] PATH_INFO: `/v3/users/e370d55247cd4e48a057d1f57c888ecf` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:39.735255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9040a-5d2b-4fbe-a774-34240053e067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e370d55247cd4e48a057d1f57c888ecf)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:39.738842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9040a-5d2b-4fbe-a774-34240053e067 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:40.019545 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0bd9040a-5d2b-4fbe-a774-34240053e067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e370d55247cd4e48a057d1f57c888ecf'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:40.030749 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0bd9040a-5d2b-4fbe-a774-34240053e067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e370d55247cd4e48a057d1f57c888ecf'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:40.092691 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0bd9040a-5d2b-4fbe-a774-34240053e067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e370d55247cd4e48a057d1f57c888ecf'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:40.094409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0bd9040a-5d2b-4fbe-a774-34240053e067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e370d55247cd4e48a057d1f57c888ecf'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:40.121464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0bd9040a-5d2b-4fbe-a774-34240053e067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e370d55247cd4e48a057d1f57c888ecf'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:40.128684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1112/2215] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:53:39 2022] DELETE /identity/v3/users/e370d55247cd4e48a057d1f57c888ecf => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:40.142772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:40.885142 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:40.889668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:40.921659 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e386ba-b2e4-4887-8c87-4fc7993d8c9c admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:41.005168 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65448552-2006-43aa-85a2-76615d678e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:41.005812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65448552-2006-43aa-85a2-76615d678e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:41.006088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65448552-2006-43aa-85a2-76615d678e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:41.006359 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65448552-2006-43aa-85a2-76615d678e5a admin admin] PATH_INFO: `/v3/projects/6aea865fe10f441bb7b41396af8750e7` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:41.007536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65448552-2006-43aa-85a2-76615d678e5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aea865fe10f441bb7b41396af8750e7)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:41.009497 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65448552-2006-43aa-85a2-76615d678e5a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:41.285142 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-65448552-2006-43aa-85a2-76615d678e5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aea865fe10f441bb7b41396af8750e7', 'request_id': 'req-65448552-2006-43aa-85a2-76615d678e5a'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:41.293217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-65448552-2006-43aa-85a2-76615d678e5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aea865fe10f441bb7b41396af8750e7', 'request_id': 'req-65448552-2006-43aa-85a2-76615d678e5a'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:41.353422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-65448552-2006-43aa-85a2-76615d678e5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aea865fe10f441bb7b41396af8750e7', 'request_id': 'req-65448552-2006-43aa-85a2-76615d678e5a'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:41.366555 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-65448552-2006-43aa-85a2-76615d678e5a admin admin] The token cache is being invalidate because project 6aea865fe10f441bb7b41396af8750e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:53:41.366972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-65448552-2006-43aa-85a2-76615d678e5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:41.367801 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-65448552-2006-43aa-85a2-76615d678e5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:53:41.369440 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1104/2216] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:53:40 2022] DELETE /identity/v3/projects/6aea865fe10f441bb7b41396af8750e7 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:41.377222 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb089b3f-2f33-4c4e-b06e-2f4e78f98f30 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:41.377740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb089b3f-2f33-4c4e-b06e-2f4e78f98f30 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:41.377872 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb089b3f-2f33-4c4e-b06e-2f4e78f98f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:41.399769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-cb089b3f-2f33-4c4e-b06e-2f4e78f98f30 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:41.399769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-cb089b3f-2f33-4c4e-b06e-2f4e78f98f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:41.477796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1113/2217] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:41.488417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f88ec8-1fe1-4b1c-a3ae-93b5f0899220 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:41.488742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f88ec8-1fe1-4b1c-a3ae-93b5f0899220 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:41.489016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f88ec8-1fe1-4b1c-a3ae-93b5f0899220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:41.510325 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d1f88ec8-1fe1-4b1c-a3ae-93b5f0899220 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:41.511796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d1f88ec8-1fe1-4b1c-a3ae-93b5f0899220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:41.521922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1105/2218] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:41.526630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb089b3f-2f33-4c4e-b06e-2f4e78f98f30 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:41.538754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb089b3f-2f33-4c4e-b06e-2f4e78f98f30 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:41.544164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481903cf-1a74-4211-b4fc-b2fc051db725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:41.544889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481903cf-1a74-4211-b4fc-b2fc051db725 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:41.545395 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481903cf-1a74-4211-b4fc-b2fc051db725 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:41.545764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481903cf-1a74-4211-b4fc-b2fc051db725 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:41.546909 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481903cf-1a74-4211-b4fc-b2fc051db725 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:53:41.547323 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481903cf-1a74-4211-b4fc-b2fc051db725 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:41.549735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481903cf-1a74-4211-b4fc-b2fc051db725 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:41.568683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1114/2219] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:53:41.570736 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f88ec8-1fe1-4b1c-a3ae-93b5f0899220 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:41.581164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f88ec8-1fe1-4b1c-a3ae-93b5f0899220 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:41.586184 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11662c8-9e2b-456a-9acc-6d1933aace50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:41.586784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11662c8-9e2b-456a-9acc-6d1933aace50 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:41.587070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11662c8-9e2b-456a-9acc-6d1933aace50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:41.587340 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11662c8-9e2b-456a-9acc-6d1933aace50 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:41.587973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11662c8-9e2b-456a-9acc-6d1933aace50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:41.590073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11662c8-9e2b-456a-9acc-6d1933aace50 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:41.669552 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1106/2220] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:41 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:41.676062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481903cf-1a74-4211-b4fc-b2fc051db725 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:41.688343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481903cf-1a74-4211-b4fc-b2fc051db725 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:41.772725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c7d24-80f7-4239-a613-3399a6bbff67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:41.773506 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c7d24-80f7-4239-a613-3399a6bbff67 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:41.773982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c7d24-80f7-4239-a613-3399a6bbff67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:41.774334 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c7d24-80f7-4239-a613-3399a6bbff67 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:41.775114 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c7d24-80f7-4239-a613-3399a6bbff67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:41.777172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c7d24-80f7-4239-a613-3399a6bbff67 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:41.810903 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:41.870973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1115/2221] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:41 2022] POST /identity/v3/users => generated 290 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:41.874122 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:41.875165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11662c8-9e2b-456a-9acc-6d1933aace50 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:41.886936 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11662c8-9e2b-456a-9acc-6d1933aace50 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:41.891834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29173a9-90de-4118-8858-063be1feeb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:41.892589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29173a9-90de-4118-8858-063be1feeb73 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:41.893027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29173a9-90de-4118-8858-063be1feeb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:41.893304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29173a9-90de-4118-8858-063be1feeb73 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:41.893952 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29173a9-90de-4118-8858-063be1feeb73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:41.895737 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29173a9-90de-4118-8858-063be1feeb73 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:41.906958 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1107/2222] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:41.911770 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c7d24-80f7-4239-a613-3399a6bbff67 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:41.922524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c7d24-80f7-4239-a613-3399a6bbff67 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:41.928830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a2fb1-a53f-4890-89a1-6c84b56fe8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:41.936179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a2fb1-a53f-4890-89a1-6c84b56fe8df admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:41.936179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a2fb1-a53f-4890-89a1-6c84b56fe8df admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:41.936179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a2fb1-a53f-4890-89a1-6c84b56fe8df admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:41.936179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a2fb1-a53f-4890-89a1-6c84b56fe8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:41.936179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a2fb1-a53f-4890-89a1-6c84b56fe8df admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:41.957655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1116/2223] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:41.959371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29173a9-90de-4118-8858-063be1feeb73 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:41.969505 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29173a9-90de-4118-8858-063be1feeb73 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:41.975148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82973f13-6d9e-44f8-8851-760264811ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:41.976008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82973f13-6d9e-44f8-8851-760264811ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:41.976292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82973f13-6d9e-44f8-8851-760264811ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:41.976759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82973f13-6d9e-44f8-8851-760264811ce9 admin admin] PATH_INFO: `/v3/projects/65b31c186fea438d9d17af25036099e4/users/9a4f3bfb16c6418cbdc1b6e8d4457a9a/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:41.992800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82973f13-6d9e-44f8-8851-760264811ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b31c186fea438d9d17af25036099e4, user_id=9a4f3bfb16c6418cbdc1b6e8d4457a9a, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:41.995436 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82973f13-6d9e-44f8-8851-760264811ce9 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:42.072848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1108/2224] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:41 2022] PUT /identity/v3/projects/65b31c186fea438d9d17af25036099e4/users/9a4f3bfb16c6418cbdc1b6e8d4457a9a/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:42.074553 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d06653-637a-4f70-ac05-230d90e1792e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:42.074846 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d06653-637a-4f70-ac05-230d90e1792e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:42.075126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d06653-637a-4f70-ac05-230d90e1792e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:42.102562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-87d06653-637a-4f70-ac05-230d90e1792e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:42.103897 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-87d06653-637a-4f70-ac05-230d90e1792e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:42.178405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1117/2225] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:42.186647 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82973f13-6d9e-44f8-8851-760264811ce9 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:42.188350 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-82973f13-6d9e-44f8-8851-760264811ce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:42.291423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82973f13-6d9e-44f8-8851-760264811ce9 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:42.295509 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7292a-a2b0-4101-8837-87333caa8564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:42.296186 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7292a-a2b0-4101-8837-87333caa8564 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:42.296537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7292a-a2b0-4101-8837-87333caa8564 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:42.296902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7292a-a2b0-4101-8837-87333caa8564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:42.313689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7292a-a2b0-4101-8837-87333caa8564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:42.316151 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7292a-a2b0-4101-8837-87333caa8564 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:42.336851 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1109/2226] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:42.863935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcb4db-d053-470a-8803-5c1b1a254ac9 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:42.865099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcb4db-d053-470a-8803-5c1b1a254ac9 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:42.865099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcb4db-d053-470a-8803-5c1b1a254ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:42.917631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-bcbcb4db-d053-470a-8803-5c1b1a254ac9 None None] MFA Rules not processed for user `be379e1a6add42b69e9fdd676efca280`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:42.920012 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-bcbcb4db-d053-470a-8803-5c1b1a254ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:43.008565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1118/2227] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 24 07:53:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:43.066963 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7292a-a2b0-4101-8837-87333caa8564 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:43.068551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-84a7292a-a2b0-4101-8837-87333caa8564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:43.170264 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7292a-a2b0-4101-8837-87333caa8564 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:43.177929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6f5b7-bf14-4fda-99b4-d62b3012bfca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b629ad3e6b9f4723b103f9845df2aaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:43.178643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6f5b7-bf14-4fda-99b4-d62b3012bfca service swift] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:43.178979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6f5b7-bf14-4fda-99b4-d62b3012bfca service swift] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:43.184043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6f5b7-bf14-4fda-99b4-d62b3012bfca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:43.190292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6f5b7-bf14-4fda-99b4-d62b3012bfca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:43.192248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6f5b7-bf14-4fda-99b4-d62b3012bfca service swift] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:43.211041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1110/2228] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 24 07:53:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:53:43.611033 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:44.111727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47aeb3-177f-4bd1-989e-b901d8db4f9d None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:44.112292 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47aeb3-177f-4bd1-989e-b901d8db4f9d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:44.113957 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47aeb3-177f-4bd1-989e-b901d8db4f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:44.161591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-eb47aeb3-177f-4bd1-989e-b901d8db4f9d None None] MFA Rules not processed for user `7f3a2203128f46528a40a9f55b54b5b9`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:44.163026 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-eb47aeb3-177f-4bd1-989e-b901d8db4f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:44.321772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1119/2229] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:44.325967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6f5b7-bf14-4fda-99b4-d62b3012bfca service swift] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:44.328888 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-7ce6f5b7-bf14-4fda-99b4-d62b3012bfca service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:44.352431 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6f5b7-bf14-4fda-99b4-d62b3012bfca service swift] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:44.359954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0faa8-ad03-4e83-bc56-7f77eb9e4979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:44.361512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0faa8-ad03-4e83-bc56-7f77eb9e4979 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:44.362055 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0faa8-ad03-4e83-bc56-7f77eb9e4979 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:44.362641 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0faa8-ad03-4e83-bc56-7f77eb9e4979 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:44.363523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0faa8-ad03-4e83-bc56-7f77eb9e4979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:44.365818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0faa8-ad03-4e83-bc56-7f77eb9e4979 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:44.384550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:44.572979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1111/2230] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:44 2022] POST /identity/v3/projects => generated 373 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:53:44.576418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47aeb3-177f-4bd1-989e-b901d8db4f9d None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:44.606592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47aeb3-177f-4bd1-989e-b901d8db4f9d None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:44.731034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5456c8a-ef9c-4dad-8a8d-e2ff26f04514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:44.731743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5456c8a-ef9c-4dad-8a8d-e2ff26f04514 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:44.732106 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5456c8a-ef9c-4dad-8a8d-e2ff26f04514 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:44.732307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5456c8a-ef9c-4dad-8a8d-e2ff26f04514 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:44.733061 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5456c8a-ef9c-4dad-8a8d-e2ff26f04514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:44.748198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5456c8a-ef9c-4dad-8a8d-e2ff26f04514 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:44.951481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1120/2231] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:44 2022] POST /identity/v3/users => generated 293 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:53:44.956246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0faa8-ad03-4e83-bc56-7f77eb9e4979 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:45.002218 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0faa8-ad03-4e83-bc56-7f77eb9e4979 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:45.023341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50218579-a820-43da-9537-f2b77d21fd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:45.026417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50218579-a820-43da-9537-f2b77d21fd41 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:45.027299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50218579-a820-43da-9537-f2b77d21fd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:45.028723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50218579-a820-43da-9537-f2b77d21fd41 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:45.030200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50218579-a820-43da-9537-f2b77d21fd41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:45.040189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50218579-a820-43da-9537-f2b77d21fd41 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:45.070070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1112/2232] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:45.074224 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5456c8a-ef9c-4dad-8a8d-e2ff26f04514 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:45.097948 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5456c8a-ef9c-4dad-8a8d-e2ff26f04514 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:45.106223 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998991c-2474-498f-b13d-1ae36e3cc8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:45.107239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998991c-2474-498f-b13d-1ae36e3cc8d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:45.110172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998991c-2474-498f-b13d-1ae36e3cc8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:45.111242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998991c-2474-498f-b13d-1ae36e3cc8d6 admin admin] PATH_INFO: `/v3/projects/036c09e638f64a5f8ec5c5b574473fb1/users/dd89b9f028ca49bfaba4ad41a3545dd8/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:45.160596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998991c-2474-498f-b13d-1ae36e3cc8d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=036c09e638f64a5f8ec5c5b574473fb1, user_id=dd89b9f028ca49bfaba4ad41a3545dd8, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:45.163328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998991c-2474-498f-b13d-1ae36e3cc8d6 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:45.266534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1121/2233] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:45 2022] PUT /identity/v3/projects/036c09e638f64a5f8ec5c5b574473fb1/users/dd89b9f028ca49bfaba4ad41a3545dd8/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:45.268743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50218579-a820-43da-9537-f2b77d21fd41 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:45.287160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50218579-a820-43da-9537-f2b77d21fd41 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:45.372082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898042f1-678a-464b-af77-4d9ce85a1f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:45.372843 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898042f1-678a-464b-af77-4d9ce85a1f9d admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:45.373002 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898042f1-678a-464b-af77-4d9ce85a1f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:45.373453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898042f1-678a-464b-af77-4d9ce85a1f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:45.374263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898042f1-678a-464b-af77-4d9ce85a1f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:45.376039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898042f1-678a-464b-af77-4d9ce85a1f9d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:45.399064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1113/2234] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:45.400268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998991c-2474-498f-b13d-1ae36e3cc8d6 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:45.418154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998991c-2474-498f-b13d-1ae36e3cc8d6 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:45.431296 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44d967-bf11-446f-b227-adbdcfa66a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:45.432295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44d967-bf11-446f-b227-adbdcfa66a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:45.432573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44d967-bf11-446f-b227-adbdcfa66a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:45.433769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44d967-bf11-446f-b227-adbdcfa66a7c admin admin] PATH_INFO: `/v3/projects/036c09e638f64a5f8ec5c5b574473fb1/users/dd89b9f028ca49bfaba4ad41a3545dd8/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:45.436680 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44d967-bf11-446f-b227-adbdcfa66a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=036c09e638f64a5f8ec5c5b574473fb1, user_id=dd89b9f028ca49bfaba4ad41a3545dd8, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:45.442013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44d967-bf11-446f-b227-adbdcfa66a7c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:45.456825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1122/2235] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:45 2022] PUT /identity/v3/projects/036c09e638f64a5f8ec5c5b574473fb1/users/dd89b9f028ca49bfaba4ad41a3545dd8/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:45.880589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:46.708004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:52.656807 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc500b7-3af0-4165-8471-d7097aea4928 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:52.657341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc500b7-3af0-4165-8471-d7097aea4928 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:52.657947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc500b7-3af0-4165-8471-d7097aea4928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:52.792054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-7fc500b7-3af0-4165-8471-d7097aea4928 None None] MFA Rules not processed for user `9a4f3bfb16c6418cbdc1b6e8d4457a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:52.793448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-7fc500b7-3af0-4165-8471-d7097aea4928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:53.008615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:53.106501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44d967-bf11-446f-b227-adbdcfa66a7c admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:53.109601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1114/2236] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:53.318657 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44d967-bf11-446f-b227-adbdcfa66a7c admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:53.330132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939fa52-a135-4035-b22c-635a9b956671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:53.331540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939fa52-a135-4035-b22c-635a9b956671 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:53.331889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939fa52-a135-4035-b22c-635a9b956671 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:53.332217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939fa52-a135-4035-b22c-635a9b956671 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:53.333475 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939fa52-a135-4035-b22c-635a9b956671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:53.338289 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939fa52-a135-4035-b22c-635a9b956671 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:53.476289 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1123/2237] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:53:53 2022] POST /identity/v3/projects => generated 375 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:53:53.479763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc500b7-3af0-4165-8471-d7097aea4928 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:53.509519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc500b7-3af0-4165-8471-d7097aea4928 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:53.653980 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a6de9-1725-4da8-a181-a93c18b006c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:53.654722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a6de9-1725-4da8-a181-a93c18b006c1 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:53.656492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a6de9-1725-4da8-a181-a93c18b006c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:53.656492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a6de9-1725-4da8-a181-a93c18b006c1 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:53.660444 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a6de9-1725-4da8-a181-a93c18b006c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:53.672340 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a6de9-1725-4da8-a181-a93c18b006c1 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:53.801009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1115/2238] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:53:53 2022] POST /identity/v3/users => generated 294 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:53:53.805179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939fa52-a135-4035-b22c-635a9b956671 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:53.845329 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939fa52-a135-4035-b22c-635a9b956671 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:53.861126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d82ebe-badb-483e-8d4c-b895124fcb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:53.863935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d82ebe-badb-483e-8d4c-b895124fcb17 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:53.864228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d82ebe-badb-483e-8d4c-b895124fcb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:53.864774 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d82ebe-badb-483e-8d4c-b895124fcb17 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:53.865442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d82ebe-badb-483e-8d4c-b895124fcb17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:53.873177 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d82ebe-badb-483e-8d4c-b895124fcb17 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:53.901667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1124/2239] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:53.905674 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a6de9-1725-4da8-a181-a93c18b006c1 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:53.920809 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a6de9-1725-4da8-a181-a93c18b006c1 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:53.940119 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca0c61-4247-4f98-bde4-d99771d1f0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:53.941120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca0c61-4247-4f98-bde4-d99771d1f0c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:53.952065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca0c61-4247-4f98-bde4-d99771d1f0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:53.952065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca0c61-4247-4f98-bde4-d99771d1f0c2 admin admin] PATH_INFO: `/v3/projects/4894bc5a1ec34931b5e4203f2c4b2544/users/fb2e0176b51b4794ae0397697c08aa96/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:53.970146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca0c61-4247-4f98-bde4-d99771d1f0c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4894bc5a1ec34931b5e4203f2c4b2544, user_id=fb2e0176b51b4794ae0397697c08aa96, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:53.973577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca0c61-4247-4f98-bde4-d99771d1f0c2 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:54.135956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1116/2240] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:53 2022] PUT /identity/v3/projects/4894bc5a1ec34931b5e4203f2c4b2544/users/fb2e0176b51b4794ae0397697c08aa96/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:53:54.142107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d82ebe-badb-483e-8d4c-b895124fcb17 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:54.156892 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d82ebe-badb-483e-8d4c-b895124fcb17 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:54.405293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a5b34-ba16-4b71-a378-d12e2e7941d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:54.406748 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a5b34-ba16-4b71-a378-d12e2e7941d6 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:54.407345 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a5b34-ba16-4b71-a378-d12e2e7941d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:54.408087 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a5b34-ba16-4b71-a378-d12e2e7941d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:54.409117 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a5b34-ba16-4b71-a378-d12e2e7941d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:54.415783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a5b34-ba16-4b71-a378-d12e2e7941d6 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:54.418690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:54.444121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1125/2241] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:53:54.449992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca0c61-4247-4f98-bde4-d99771d1f0c2 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:54.474814 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca0c61-4247-4f98-bde4-d99771d1f0c2 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:54.491929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35df89-bc5d-44b6-8799-103e1f19a56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:54.493334 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35df89-bc5d-44b6-8799-103e1f19a56f admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:54.493859 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35df89-bc5d-44b6-8799-103e1f19a56f admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:54.494675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35df89-bc5d-44b6-8799-103e1f19a56f admin admin] PATH_INFO: `/v3/projects/4894bc5a1ec34931b5e4203f2c4b2544/users/fb2e0176b51b4794ae0397697c08aa96/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:54.506663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35df89-bc5d-44b6-8799-103e1f19a56f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4894bc5a1ec34931b5e4203f2c4b2544, user_id=fb2e0176b51b4794ae0397697c08aa96, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:54.520109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35df89-bc5d-44b6-8799-103e1f19a56f admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:54.552818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1117/2242] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:53:54 2022] PUT /identity/v3/projects/4894bc5a1ec34931b5e4203f2c4b2544/users/fb2e0176b51b4794ae0397697c08aa96/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:53:57.515519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba08f1d-d016-4f61-9574-a8f6b8831760 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:57.516242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba08f1d-d016-4f61-9574-a8f6b8831760 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:57.516242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba08f1d-d016-4f61-9574-a8f6b8831760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:57.631463 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-5ba08f1d-d016-4f61-9574-a8f6b8831760 None None] MFA Rules not processed for user `dd89b9f028ca49bfaba4ad41a3545dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:57.632865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5ba08f1d-d016-4f61-9574-a8f6b8831760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:57.886065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1126/2243] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:53:57.901326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35df89-bc5d-44b6-8799-103e1f19a56f admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:57.903469 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-7a35df89-bc5d-44b6-8799-103e1f19a56f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:58.147726 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35df89-bc5d-44b6-8799-103e1f19a56f admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:58.163531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abb398-24f2-474d-91ed-a8ba1137a767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:58.168501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abb398-24f2-474d-91ed-a8ba1137a767 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:58.170910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abb398-24f2-474d-91ed-a8ba1137a767 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:58.172678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abb398-24f2-474d-91ed-a8ba1137a767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:58.249597 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:58.379471 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abb398-24f2-474d-91ed-a8ba1137a767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:58.381818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abb398-24f2-474d-91ed-a8ba1137a767 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:58.414581 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1118/2244] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:58.784947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a51e6d-ee42-4977-b1fa-35d4c947bc2a None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:58.786108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a51e6d-ee42-4977-b1fa-35d4c947bc2a None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:58.786603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a51e6d-ee42-4977-b1fa-35d4c947bc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:58.824402 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-b3a51e6d-ee42-4977-b1fa-35d4c947bc2a None None] MFA Rules not processed for user `aee01beeba5b48c3b6c25489743de40e`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:58.829113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b3a51e6d-ee42-4977-b1fa-35d4c947bc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:58.949805 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1127/2245] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:53:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:00.283111 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea79bf8-1623-46d5-b170-1896193d91b0 None None] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:00.283709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea79bf8-1623-46d5-b170-1896193d91b0 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:00.283709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea79bf8-1623-46d5-b170-1896193d91b0 None None] PATH_INFO: `/` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:00.285323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1119/2246] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 24 07:54:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:54:00.327029 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122babde-fdf7-4c4d-bb16-129140f4dffa None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:00.328076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122babde-fdf7-4c4d-bb16-129140f4dffa None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:00.329638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122babde-fdf7-4c4d-bb16-129140f4dffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:00.376421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-122babde-fdf7-4c4d-bb16-129140f4dffa None None] MFA Rules not processed for user `ea3816d5ae054741ac2c5a2eec3eb617`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:00.378138 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-122babde-fdf7-4c4d-bb16-129140f4dffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:00.606693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1128/2247] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 24 07:54:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:00.656861 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea79bf8-1623-46d5-b170-1896193d91b0 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:00.662630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-bea79bf8-1623-46d5-b170-1896193d91b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:00.710953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea79bf8-1623-46d5-b170-1896193d91b0 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:00.726446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37724cc-8176-4ebf-888d-8c77e5ad951c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:00.727308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37724cc-8176-4ebf-888d-8c77e5ad951c service placement] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:00.727751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37724cc-8176-4ebf-888d-8c77e5ad951c service placement] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:00.728195 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37724cc-8176-4ebf-888d-8c77e5ad951c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:00.760926 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37724cc-8176-4ebf-888d-8c77e5ad951c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:00.762927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37724cc-8176-4ebf-888d-8c77e5ad951c service placement] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:00.802491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1120/2248] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:54:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:54:02.998744 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122babde-fdf7-4c4d-bb16-129140f4dffa None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:03.000218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-122babde-fdf7-4c4d-bb16-129140f4dffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:03.080207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:03.181451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122babde-fdf7-4c4d-bb16-129140f4dffa None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:03.190230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac946d-5cee-4478-8680-2438ca589682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:03.190891 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac946d-5cee-4478-8680-2438ca589682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:03.191229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac946d-5cee-4478-8680-2438ca589682 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:03.191513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac946d-5cee-4478-8680-2438ca589682 admin admin] PATH_INFO: `/v3/users/1951f743398e4b3da175b29819d7c64b` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:03.192839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac946d-5cee-4478-8680-2438ca589682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1951f743398e4b3da175b29819d7c64b)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:03.196496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac946d-5cee-4478-8680-2438ca589682 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:03.516084 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-efac946d-5cee-4478-8680-2438ca589682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1951f743398e4b3da175b29819d7c64b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:03.559078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-efac946d-5cee-4478-8680-2438ca589682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1951f743398e4b3da175b29819d7c64b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:03.633570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-efac946d-5cee-4478-8680-2438ca589682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1951f743398e4b3da175b29819d7c64b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:03.634391 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-efac946d-5cee-4478-8680-2438ca589682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1951f743398e4b3da175b29819d7c64b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:03.671596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-efac946d-5cee-4478-8680-2438ca589682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1951f743398e4b3da175b29819d7c64b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:03.694896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1129/2249] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:54:02 2022] DELETE /identity/v3/users/1951f743398e4b3da175b29819d7c64b => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:04.685528 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37724cc-8176-4ebf-888d-8c77e5ad951c service placement] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:04.705543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37724cc-8176-4ebf-888d-8c77e5ad951c service placement] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:04.806039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:04.807153 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:04.807662 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:04.808321 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] PATH_INFO: `/v3/projects/5221c073fdfb4d10adc38d6a18cf7841` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:04.809881 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5221c073fdfb4d10adc38d6a18cf7841)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:04.812966 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:04.960691 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5221c073fdfb4d10adc38d6a18cf7841', 'request_id': 'req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:04.982410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5221c073fdfb4d10adc38d6a18cf7841', 'request_id': 'req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:05.056365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5221c073fdfb4d10adc38d6a18cf7841', 'request_id': 'req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:05.070715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] The token cache is being invalidate because project 5221c073fdfb4d10adc38d6a18cf7841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:54:05.071007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:05.077936 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:05.079887 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1121/2250] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:54:04 2022] DELETE /identity/v3/projects/5221c073fdfb4d10adc38d6a18cf7841 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:05.083952 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac946d-5cee-4478-8680-2438ca589682 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:05.085932 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-efac946d-5cee-4478-8680-2438ca589682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:05.111803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac946d-5cee-4478-8680-2438ca589682 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:05.270496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acd823-cae2-4a9d-be73-b9f6d650b908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:05.271207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acd823-cae2-4a9d-be73-b9f6d650b908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:05.271503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acd823-cae2-4a9d-be73-b9f6d650b908 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:05.272215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acd823-cae2-4a9d-be73-b9f6d650b908 admin admin] PATH_INFO: `/v3/users/0f668737502f4e8fb7c97c0cadc1befa` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:05.273980 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acd823-cae2-4a9d-be73-b9f6d650b908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f668737502f4e8fb7c97c0cadc1befa)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:05.278360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acd823-cae2-4a9d-be73-b9f6d650b908 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:05.588129 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:05.596853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b2acd823-cae2-4a9d-be73-b9f6d650b908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f668737502f4e8fb7c97c0cadc1befa'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:05.618100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b2acd823-cae2-4a9d-be73-b9f6d650b908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f668737502f4e8fb7c97c0cadc1befa'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:05.742862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b2acd823-cae2-4a9d-be73-b9f6d650b908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f668737502f4e8fb7c97c0cadc1befa'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:05.743684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b2acd823-cae2-4a9d-be73-b9f6d650b908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f668737502f4e8fb7c97c0cadc1befa'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:05.773265 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b2acd823-cae2-4a9d-be73-b9f6d650b908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f668737502f4e8fb7c97c0cadc1befa'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:05.797797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1130/2251] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:54:05 2022] DELETE /identity/v3/users/0f668737502f4e8fb7c97c0cadc1befa => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:06.612310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:06.614869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:06.675779 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d6e8e-ead7-4dd0-8f7a-ff034658ef28 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:06.903628 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcc5af-2a86-4477-99d2-67fa06a9a98f None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:06.904500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcc5af-2a86-4477-99d2-67fa06a9a98f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:06.905692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:06.906324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcc5af-2a86-4477-99d2-67fa06a9a98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:06.927038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbfe37c-f863-4fd3-bce1-6e5167812b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:06.927992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfe37c-f863-4fd3-bce1-6e5167812b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:06.928449 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfe37c-f863-4fd3-bce1-6e5167812b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:06.929011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfe37c-f863-4fd3-bce1-6e5167812b53 admin admin] PATH_INFO: `/v3/projects/fd8b2b88b1cf4ee6b05596ae9f30431d` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:06.930907 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbfe37c-f863-4fd3-bce1-6e5167812b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd8b2b88b1cf4ee6b05596ae9f30431d)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:06.934615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbfe37c-f863-4fd3-bce1-6e5167812b53 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:06.943708 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-36bcc5af-2a86-4477-99d2-67fa06a9a98f None None] MFA Rules not processed for user `7f3a2203128f46528a40a9f55b54b5b9`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:06.953273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-36bcc5af-2a86-4477-99d2-67fa06a9a98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:07.106859 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8cbfe37c-f863-4fd3-bce1-6e5167812b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd8b2b88b1cf4ee6b05596ae9f30431d', 'request_id': 'req-8cbfe37c-f863-4fd3-bce1-6e5167812b53'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:07.115883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8cbfe37c-f863-4fd3-bce1-6e5167812b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd8b2b88b1cf4ee6b05596ae9f30431d', 'request_id': 'req-8cbfe37c-f863-4fd3-bce1-6e5167812b53'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:07.193860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8cbfe37c-f863-4fd3-bce1-6e5167812b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd8b2b88b1cf4ee6b05596ae9f30431d', 'request_id': 'req-8cbfe37c-f863-4fd3-bce1-6e5167812b53'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:07.220571 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8cbfe37c-f863-4fd3-bce1-6e5167812b53 admin admin] The token cache is being invalidate because project fd8b2b88b1cf4ee6b05596ae9f30431d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:54:07.220964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8cbfe37c-f863-4fd3-bce1-6e5167812b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:07.224055 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8cbfe37c-f863-4fd3-bce1-6e5167812b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:07.230028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1122/2252] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:54:06 2022] DELETE /identity/v3/projects/fd8b2b88b1cf4ee6b05596ae9f30431d => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:07.234826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e5db3-1d50-4d65-a9c7-d013a418f080 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:07.235745 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e5db3-1d50-4d65-a9c7-d013a418f080 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:07.239515 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e5db3-1d50-4d65-a9c7-d013a418f080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:07.258742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1131/2253] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:07.309526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-5e8e5db3-1d50-4d65-a9c7-d013a418f080 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:07.313077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5e8e5db3-1d50-4d65-a9c7-d013a418f080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:07.535651 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1123/2254] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:07.543253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d93fe2-5b7a-447e-8cf9-82341ba612be None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:07.543762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d93fe2-5b7a-447e-8cf9-82341ba612be None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:07.545083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d93fe2-5b7a-447e-8cf9-82341ba612be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:07.596143 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-c1d93fe2-5b7a-447e-8cf9-82341ba612be None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:07.597426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c1d93fe2-5b7a-447e-8cf9-82341ba612be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:07.618140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1132/2255] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:07.626475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e5db3-1d50-4d65-a9c7-d013a418f080 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:07.639021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e5db3-1d50-4d65-a9c7-d013a418f080 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:07.655224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ea892-fa08-4481-809c-b312f2b22d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:07.656067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ea892-fa08-4481-809c-b312f2b22d6f admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:07.657831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ea892-fa08-4481-809c-b312f2b22d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:07.658302 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ea892-fa08-4481-809c-b312f2b22d6f admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:07.659117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ea892-fa08-4481-809c-b312f2b22d6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:54:07.659605 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ea892-fa08-4481-809c-b312f2b22d6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:07.661659 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ea892-fa08-4481-809c-b312f2b22d6f admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:07.686995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1124/2256] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:54:07.692374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d93fe2-5b7a-447e-8cf9-82341ba612be None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:07.709134 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d93fe2-5b7a-447e-8cf9-82341ba612be None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:07.721477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc70af-47d1-49bc-abb9-2d447d5d4ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:07.722120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc70af-47d1-49bc-abb9-2d447d5d4ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:07.722404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc70af-47d1-49bc-abb9-2d447d5d4ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:07.722675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc70af-47d1-49bc-abb9-2d447d5d4ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:07.727477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc70af-47d1-49bc-abb9-2d447d5d4ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:07.734853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc70af-47d1-49bc-abb9-2d447d5d4ec5 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:07.895735 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1133/2257] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:54:07 2022] POST /identity/v3/projects => generated 377 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:54:07.897328 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ea892-fa08-4481-809c-b312f2b22d6f admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:07.908495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ea892-fa08-4481-809c-b312f2b22d6f admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:08.069372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea7789-03a5-484d-9003-1d3879b23d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:08.070001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea7789-03a5-484d-9003-1d3879b23d29 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:08.070284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea7789-03a5-484d-9003-1d3879b23d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:08.070727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea7789-03a5-484d-9003-1d3879b23d29 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:08.071541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea7789-03a5-484d-9003-1d3879b23d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:08.073510 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea7789-03a5-484d-9003-1d3879b23d29 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:08.128734 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:08.227085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1125/2258] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:54:07 2022] POST /identity/v3/users => generated 289 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:54:08.228953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc70af-47d1-49bc-abb9-2d447d5d4ec5 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:08.244053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc70af-47d1-49bc-abb9-2d447d5d4ec5 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:08.253196 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0dcba-cba3-4be0-8a66-911bbf19e385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:08.254070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0dcba-cba3-4be0-8a66-911bbf19e385 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:08.254560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0dcba-cba3-4be0-8a66-911bbf19e385 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:08.255048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0dcba-cba3-4be0-8a66-911bbf19e385 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:08.255853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c0dcba-cba3-4be0-8a66-911bbf19e385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:08.258526 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c0dcba-cba3-4be0-8a66-911bbf19e385 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:08.289753 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1134/2259] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:54:08.293491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea7789-03a5-484d-9003-1d3879b23d29 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:08.304924 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea7789-03a5-484d-9003-1d3879b23d29 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:08.319527 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a9bdc-f454-4c6a-98e3-99aa13be5cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:08.320548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a9bdc-f454-4c6a-98e3-99aa13be5cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:08.321020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a9bdc-f454-4c6a-98e3-99aa13be5cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:08.321545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a9bdc-f454-4c6a-98e3-99aa13be5cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:08.322348 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a9bdc-f454-4c6a-98e3-99aa13be5cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:08.324275 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a9bdc-f454-4c6a-98e3-99aa13be5cc1 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:08.359381 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1126/2260] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:54:08.361746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0dcba-cba3-4be0-8a66-911bbf19e385 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:08.397633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0dcba-cba3-4be0-8a66-911bbf19e385 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:08.416218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7eeffb-2020-42ec-bdce-75952973ea98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:08.418167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eeffb-2020-42ec-bdce-75952973ea98 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:08.418543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eeffb-2020-42ec-bdce-75952973ea98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:08.418752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eeffb-2020-42ec-bdce-75952973ea98 admin admin] PATH_INFO: `/v3/projects/917f19638f1e4c96b6ecf30de3faeca4/users/c8ad14730db741c9bf44fd31a152678b/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:08.490738 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7eeffb-2020-42ec-bdce-75952973ea98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=917f19638f1e4c96b6ecf30de3faeca4, user_id=c8ad14730db741c9bf44fd31a152678b, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:08.498896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7eeffb-2020-42ec-bdce-75952973ea98 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:08.635756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1135/2261] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:54:08 2022] PUT /identity/v3/projects/917f19638f1e4c96b6ecf30de3faeca4/users/c8ad14730db741c9bf44fd31a152678b/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:08.647817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc5e94-8562-4984-be54-53ba17b6192c None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:08.648608 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc5e94-8562-4984-be54-53ba17b6192c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:08.649054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc5e94-8562-4984-be54-53ba17b6192c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:08.665164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4e648-05df-436f-98b4-65f9cbf296ab None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:08.665655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4e648-05df-436f-98b4-65f9cbf296ab None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:08.666147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4e648-05df-436f-98b4-65f9cbf296ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:08.707906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d7dc5e94-8562-4984-be54-53ba17b6192c None None] MFA Rules not processed for user `c8ad14730db741c9bf44fd31a152678b`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:08.709244 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d7dc5e94-8562-4984-be54-53ba17b6192c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:08.740244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-cff4e648-05df-436f-98b4-65f9cbf296ab None None] MFA Rules not processed for user `fb2e0176b51b4794ae0397697c08aa96`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:08.746801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-cff4e648-05df-436f-98b4-65f9cbf296ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:08.868532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1127/2262] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:08.884923 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc5e94-8562-4984-be54-53ba17b6192c None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:08.886767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d7dc5e94-8562-4984-be54-53ba17b6192c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:08.888801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:08.945670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1136/2263] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:08.958657 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4e648-05df-436f-98b4-65f9cbf296ab None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:08.960274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-cff4e648-05df-436f-98b4-65f9cbf296ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:08.990153 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc5e94-8562-4984-be54-53ba17b6192c None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:08.995362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a73e97-4b58-4d16-a5d4-4151305ccc08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:08.996745 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a73e97-4b58-4d16-a5d4-4151305ccc08 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:08.997038 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a73e97-4b58-4d16-a5d4-4151305ccc08 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:08.997329 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a73e97-4b58-4d16-a5d4-4151305ccc08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:09.011258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a73e97-4b58-4d16-a5d4-4151305ccc08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:09.013295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a73e97-4b58-4d16-a5d4-4151305ccc08 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:09.024968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1128/2264] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:54:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:54:09.168632 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4e648-05df-436f-98b4-65f9cbf296ab None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:09.185389 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ecd56-2c30-4656-99fb-af920138bae8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:09.185733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ecd56-2c30-4656-99fb-af920138bae8 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:09.186021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ecd56-2c30-4656-99fb-af920138bae8 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:09.186301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ecd56-2c30-4656-99fb-af920138bae8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:09.197690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-bc7ecd56-2c30-4656-99fb-af920138bae8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:09.353823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ecd56-2c30-4656-99fb-af920138bae8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:09.357832 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ecd56-2c30-4656-99fb-af920138bae8 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:09.376604 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1137/2265] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:54:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:54:13.523927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f94e16-be5c-4f56-b27c-477bd68fc7a5 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:13.524634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f94e16-be5c-4f56-b27c-477bd68fc7a5 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:13.524634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f94e16-be5c-4f56-b27c-477bd68fc7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:13.585865 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-93f94e16-be5c-4f56-b27c-477bd68fc7a5 None None] MFA Rules not processed for user `e23edadcfe3d4a0b81b13e278682ca2f`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:13.587280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-93f94e16-be5c-4f56-b27c-477bd68fc7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:13.731647 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1129/2266] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:19.370787 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=62 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:20.527995 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c97be8-50f0-494c-ae6c-c63f261153f9 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:20.528566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c97be8-50f0-494c-ae6c-c63f261153f9 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:20.528566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c97be8-50f0-494c-ae6c-c63f261153f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:20.548364 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a9c97be8-50f0-494c-ae6c-c63f261153f9 None None] MFA Rules not processed for user `be379e1a6add42b69e9fdd676efca280`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:20.549811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a9c97be8-50f0-494c-ae6c-c63f261153f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:20.622700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1138/2267] 10.222.0.26 () {58 vars in 967 bytes} [Wed Aug 24 07:54:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:20.636567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f94e16-be5c-4f56-b27c-477bd68fc7a5 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:20.638133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-93f94e16-be5c-4f56-b27c-477bd68fc7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:20.666324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:20.745804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f94e16-be5c-4f56-b27c-477bd68fc7a5 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:20.750929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108461e-65d9-4147-8aeb-db4a939e2f64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b629ad3e6b9f4723b103f9845df2aaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:20.755394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108461e-65d9-4147-8aeb-db4a939e2f64 service swift] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:20.755394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108461e-65d9-4147-8aeb-db4a939e2f64 service swift] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:20.755394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108461e-65d9-4147-8aeb-db4a939e2f64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:20.761639 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108461e-65d9-4147-8aeb-db4a939e2f64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:20.763258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108461e-65d9-4147-8aeb-db4a939e2f64 service swift] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:20.772823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1130/2268] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 24 07:54:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:54:24.970852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c97be8-50f0-494c-ae6c-c63f261153f9 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:24.970852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a9c97be8-50f0-494c-ae6c-c63f261153f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:25.064633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c97be8-50f0-494c-ae6c-c63f261153f9 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:25.070325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17474af7-f668-4d4f-ba67-ede1ae97b92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:25.071075 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17474af7-f668-4d4f-ba67-ede1ae97b92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:25.071465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17474af7-f668-4d4f-ba67-ede1ae97b92c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:25.072177 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17474af7-f668-4d4f-ba67-ede1ae97b92c admin admin] PATH_INFO: `/v3/users/83fa9e38ff6d4620aae391266c82fa1c` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:25.073210 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17474af7-f668-4d4f-ba67-ede1ae97b92c admin admin] RBAC: Authorizing `identity:delete_user(user_id=83fa9e38ff6d4620aae391266c82fa1c)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:25.080328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17474af7-f668-4d4f-ba67-ede1ae97b92c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:25.320017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-17474af7-f668-4d4f-ba67-ede1ae97b92c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83fa9e38ff6d4620aae391266c82fa1c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:25.326974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-17474af7-f668-4d4f-ba67-ede1ae97b92c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83fa9e38ff6d4620aae391266c82fa1c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:25.433255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-17474af7-f668-4d4f-ba67-ede1ae97b92c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83fa9e38ff6d4620aae391266c82fa1c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:25.433918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-17474af7-f668-4d4f-ba67-ede1ae97b92c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83fa9e38ff6d4620aae391266c82fa1c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:25.447211 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-17474af7-f668-4d4f-ba67-ede1ae97b92c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83fa9e38ff6d4620aae391266c82fa1c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:25.453459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1139/2269] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:54:24 2022] DELETE /identity/v3/users/83fa9e38ff6d4620aae391266c82fa1c => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:25.725278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108461e-65d9-4147-8aeb-db4a939e2f64 service swift] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:25.738695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108461e-65d9-4147-8aeb-db4a939e2f64 service swift] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:25.801001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60d660-0f26-4817-896d-975f0e4311bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:25.801634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60d660-0f26-4817-896d-975f0e4311bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:25.801914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60d660-0f26-4817-896d-975f0e4311bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:25.802191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60d660-0f26-4817-896d-975f0e4311bf admin admin] PATH_INFO: `/v3/projects/4decf63db0594949b321e3450a122100` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:25.803323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60d660-0f26-4817-896d-975f0e4311bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4decf63db0594949b321e3450a122100)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:25.805633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60d660-0f26-4817-896d-975f0e4311bf admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:25.996295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ac60d660-0f26-4817-896d-975f0e4311bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4decf63db0594949b321e3450a122100', 'request_id': 'req-ac60d660-0f26-4817-896d-975f0e4311bf'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:26.006975 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ac60d660-0f26-4817-896d-975f0e4311bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4decf63db0594949b321e3450a122100', 'request_id': 'req-ac60d660-0f26-4817-896d-975f0e4311bf'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:26.084280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ac60d660-0f26-4817-896d-975f0e4311bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4decf63db0594949b321e3450a122100', 'request_id': 'req-ac60d660-0f26-4817-896d-975f0e4311bf'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:26.099368 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ac60d660-0f26-4817-896d-975f0e4311bf admin admin] The token cache is being invalidate because project 4decf63db0594949b321e3450a122100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:54:26.100117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ac60d660-0f26-4817-896d-975f0e4311bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:26.101172 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ac60d660-0f26-4817-896d-975f0e4311bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:26.102943 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1131/2270] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:54:25 2022] DELETE /identity/v3/projects/4decf63db0594949b321e3450a122100 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:26.108369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4d266-978b-4089-8fe2-e70a14a2c449 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:26.108369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4d266-978b-4089-8fe2-e70a14a2c449 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:26.108876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4d266-978b-4089-8fe2-e70a14a2c449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:26.134098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-91e4d266-978b-4089-8fe2-e70a14a2c449 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:26.135314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-91e4d266-978b-4089-8fe2-e70a14a2c449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:26.160332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:26.213943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1140/2271] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:54:26.222380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd04c5-5636-4430-a9cf-db3224bd4ec1 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:26.222593 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd04c5-5636-4430-a9cf-db3224bd4ec1 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:26.222876 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd04c5-5636-4430-a9cf-db3224bd4ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:26.252244 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-6bbd04c5-5636-4430-a9cf-db3224bd4ec1 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:26.254129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6bbd04c5-5636-4430-a9cf-db3224bd4ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:26.278736 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1132/2272] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:26.280351 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4d266-978b-4089-8fe2-e70a14a2c449 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:26.292178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4d266-978b-4089-8fe2-e70a14a2c449 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:26.298576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0f314-92db-4ed6-ab89-ed6ed00cf127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:26.299192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0f314-92db-4ed6-ab89-ed6ed00cf127 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:26.299474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0f314-92db-4ed6-ab89-ed6ed00cf127 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:26.299743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0f314-92db-4ed6-ab89-ed6ed00cf127 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:26.300394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0f314-92db-4ed6-ab89-ed6ed00cf127 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:54:26.300716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0f314-92db-4ed6-ab89-ed6ed00cf127 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:26.303097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0f314-92db-4ed6-ab89-ed6ed00cf127 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:26.321358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1141/2273] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:54:26.322525 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd04c5-5636-4430-a9cf-db3224bd4ec1 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:26.336251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd04c5-5636-4430-a9cf-db3224bd4ec1 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:26.341858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e92852-38ce-4a7f-8d07-da6fdf751e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:26.342464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e92852-38ce-4a7f-8d07-da6fdf751e26 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:26.343370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e92852-38ce-4a7f-8d07-da6fdf751e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:26.343655 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e92852-38ce-4a7f-8d07-da6fdf751e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:26.344788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e92852-38ce-4a7f-8d07-da6fdf751e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:26.347864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e92852-38ce-4a7f-8d07-da6fdf751e26 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:26.441566 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1133/2274] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:54:26 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:54:26.445574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0f314-92db-4ed6-ab89-ed6ed00cf127 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:26.456699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0f314-92db-4ed6-ab89-ed6ed00cf127 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:26.514264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3d01c-a187-45d7-b3b4-e4d38245ce12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:26.514792 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3d01c-a187-45d7-b3b4-e4d38245ce12 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:26.515074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3d01c-a187-45d7-b3b4-e4d38245ce12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:26.515388 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3d01c-a187-45d7-b3b4-e4d38245ce12 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:26.516023 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3d01c-a187-45d7-b3b4-e4d38245ce12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:26.518712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3d01c-a187-45d7-b3b4-e4d38245ce12 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:26.633769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1142/2275] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:54:26 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:54:26.634547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e92852-38ce-4a7f-8d07-da6fdf751e26 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:26.646921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e92852-38ce-4a7f-8d07-da6fdf751e26 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:26.652085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701f7bf-95a6-4f99-b3ac-91e22ae92fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:26.653748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701f7bf-95a6-4f99-b3ac-91e22ae92fce admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:26.653748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701f7bf-95a6-4f99-b3ac-91e22ae92fce admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:26.653748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701f7bf-95a6-4f99-b3ac-91e22ae92fce admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:26.654019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701f7bf-95a6-4f99-b3ac-91e22ae92fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:26.655760 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701f7bf-95a6-4f99-b3ac-91e22ae92fce admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:26.658258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:26.670053 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1134/2276] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:54:26.676008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3d01c-a187-45d7-b3b4-e4d38245ce12 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:26.686132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3d01c-a187-45d7-b3b4-e4d38245ce12 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:26.691768 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa959f4-4379-43c2-a13b-8329bc227d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:26.692384 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa959f4-4379-43c2-a13b-8329bc227d8f admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:26.692733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa959f4-4379-43c2-a13b-8329bc227d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:26.692979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa959f4-4379-43c2-a13b-8329bc227d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:26.693657 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa959f4-4379-43c2-a13b-8329bc227d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:26.695272 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa959f4-4379-43c2-a13b-8329bc227d8f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:26.707136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1143/2277] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:54:26.711051 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701f7bf-95a6-4f99-b3ac-91e22ae92fce admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:26.720994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701f7bf-95a6-4f99-b3ac-91e22ae92fce admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:26.725865 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd59acc-061c-46cc-bdb0-1dce6d3ec311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:26.726572 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd59acc-061c-46cc-bdb0-1dce6d3ec311 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:26.726857 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd59acc-061c-46cc-bdb0-1dce6d3ec311 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:26.727136 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd59acc-061c-46cc-bdb0-1dce6d3ec311 admin admin] PATH_INFO: `/v3/projects/55b3a9722ed54de0a518c434bdf65fa0/users/68860ffbd71d4e698ce099690c2b456b/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:26.741721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd59acc-061c-46cc-bdb0-1dce6d3ec311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55b3a9722ed54de0a518c434bdf65fa0, user_id=68860ffbd71d4e698ce099690c2b456b, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:26.744394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd59acc-061c-46cc-bdb0-1dce6d3ec311 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:26.809960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1135/2278] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:54:26 2022] PUT /identity/v3/projects/55b3a9722ed54de0a518c434bdf65fa0/users/68860ffbd71d4e698ce099690c2b456b/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:26.821551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b85770-2709-4196-9ac9-6cfe5a17cdfe None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:26.821870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b85770-2709-4196-9ac9-6cfe5a17cdfe None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:26.822153 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b85770-2709-4196-9ac9-6cfe5a17cdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:26.854731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-99b85770-2709-4196-9ac9-6cfe5a17cdfe None None] MFA Rules not processed for user `68860ffbd71d4e698ce099690c2b456b`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:26.856023 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-99b85770-2709-4196-9ac9-6cfe5a17cdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:26.941154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1144/2279] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:26.953461 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd59acc-061c-46cc-bdb0-1dce6d3ec311 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:26.964739 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-dbd59acc-061c-46cc-bdb0-1dce6d3ec311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:27.036210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd59acc-061c-46cc-bdb0-1dce6d3ec311 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:27.040071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5a773-3f98-4ea8-aacb-f9e57d1aa92b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:27.040996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5a773-3f98-4ea8-aacb-f9e57d1aa92b service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:27.041424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5a773-3f98-4ea8-aacb-f9e57d1aa92b service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:27.041870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5a773-3f98-4ea8-aacb-f9e57d1aa92b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:27.051974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5a773-3f98-4ea8-aacb-f9e57d1aa92b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:27.054468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5a773-3f98-4ea8-aacb-f9e57d1aa92b service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:27.066930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1136/2280] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:54:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:54:31.384850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b85770-2709-4196-9ac9-6cfe5a17cdfe None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:31.405905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:31.448324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b85770-2709-4196-9ac9-6cfe5a17cdfe None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:31.453093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7c9e6-a863-47ac-be54-3f96abb21fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:31.453775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7c9e6-a863-47ac-be54-3f96abb21fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:31.454063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7c9e6-a863-47ac-be54-3f96abb21fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:31.454851 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7c9e6-a863-47ac-be54-3f96abb21fb0 admin admin] PATH_INFO: `/v3/users/68860ffbd71d4e698ce099690c2b456b` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:31.456097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7c9e6-a863-47ac-be54-3f96abb21fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68860ffbd71d4e698ce099690c2b456b)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:31.458078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7c9e6-a863-47ac-be54-3f96abb21fb0 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:31.681512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0de7c9e6-a863-47ac-be54-3f96abb21fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68860ffbd71d4e698ce099690c2b456b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:31.688366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0de7c9e6-a863-47ac-be54-3f96abb21fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68860ffbd71d4e698ce099690c2b456b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:31.792608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0de7c9e6-a863-47ac-be54-3f96abb21fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68860ffbd71d4e698ce099690c2b456b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:31.798256 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0de7c9e6-a863-47ac-be54-3f96abb21fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68860ffbd71d4e698ce099690c2b456b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:31.809278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0de7c9e6-a863-47ac-be54-3f96abb21fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68860ffbd71d4e698ce099690c2b456b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:31.815634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1145/2281] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:54:31 2022] DELETE /identity/v3/users/68860ffbd71d4e698ce099690c2b456b => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:31.820960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040b765-5c96-4292-bd30-238233d3dd99 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:31.820960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040b765-5c96-4292-bd30-238233d3dd99 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:31.820960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040b765-5c96-4292-bd30-238233d3dd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:31.844989 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-0040b765-5c96-4292-bd30-238233d3dd99 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:31.850542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0040b765-5c96-4292-bd30-238233d3dd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:31.913220 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1137/2282] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:31.923206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7c9e6-a863-47ac-be54-3f96abb21fb0 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:31.925377 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0de7c9e6-a863-47ac-be54-3f96abb21fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:31.996741 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7c9e6-a863-47ac-be54-3f96abb21fb0 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:32.000345 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d67a3-6d09-4a98-b68a-f8385b7375ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:32.000990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d67a3-6d09-4a98-b68a-f8385b7375ce service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:32.001325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d67a3-6d09-4a98-b68a-f8385b7375ce service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:32.001599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d67a3-6d09-4a98-b68a-f8385b7375ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:32.010728 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d67a3-6d09-4a98-b68a-f8385b7375ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:32.012158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d67a3-6d09-4a98-b68a-f8385b7375ce service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:32.021970 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1146/2283] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:54:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:54:32.510943 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040b765-5c96-4292-bd30-238233d3dd99 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:32.522089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0040b765-5c96-4292-bd30-238233d3dd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:32.542411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040b765-5c96-4292-bd30-238233d3dd99 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:32.547354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35357194-613f-4df2-a0f4-35f7e4e4dbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:32.547986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35357194-613f-4df2-a0f4-35f7e4e4dbdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:32.548285 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35357194-613f-4df2-a0f4-35f7e4e4dbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:32.548558 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35357194-613f-4df2-a0f4-35f7e4e4dbdc admin admin] PATH_INFO: `/v3/projects/55b3a9722ed54de0a518c434bdf65fa0` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:32.550171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35357194-613f-4df2-a0f4-35f7e4e4dbdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=55b3a9722ed54de0a518c434bdf65fa0)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:32.552347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35357194-613f-4df2-a0f4-35f7e4e4dbdc admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:32.662772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-35357194-613f-4df2-a0f4-35f7e4e4dbdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55b3a9722ed54de0a518c434bdf65fa0', 'request_id': 'req-35357194-613f-4df2-a0f4-35f7e4e4dbdc'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:32.669334 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-35357194-613f-4df2-a0f4-35f7e4e4dbdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55b3a9722ed54de0a518c434bdf65fa0', 'request_id': 'req-35357194-613f-4df2-a0f4-35f7e4e4dbdc'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:32.774655 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-35357194-613f-4df2-a0f4-35f7e4e4dbdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55b3a9722ed54de0a518c434bdf65fa0', 'request_id': 'req-35357194-613f-4df2-a0f4-35f7e4e4dbdc'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:32.781583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:32.786547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-35357194-613f-4df2-a0f4-35f7e4e4dbdc admin admin] The token cache is being invalidate because project 55b3a9722ed54de0a518c434bdf65fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:54:32.786840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-35357194-613f-4df2-a0f4-35f7e4e4dbdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:32.787472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-35357194-613f-4df2-a0f4-35f7e4e4dbdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:32.789062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1138/2284] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:54:32 2022] DELETE /identity/v3/projects/55b3a9722ed54de0a518c434bdf65fa0 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:32.793968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8705543-3ad9-4dfd-ad91-f24eb991436e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:32.794489 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8705543-3ad9-4dfd-ad91-f24eb991436e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:32.794592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8705543-3ad9-4dfd-ad91-f24eb991436e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:32.814222 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-e8705543-3ad9-4dfd-ad91-f24eb991436e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:32.815446 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e8705543-3ad9-4dfd-ad91-f24eb991436e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:32.855170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:32.877281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1147/2285] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:32.885662 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d42ae-40cb-4a90-843d-4c6476a4c59f None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:32.888369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d42ae-40cb-4a90-843d-4c6476a4c59f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:32.888369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d42ae-40cb-4a90-843d-4c6476a4c59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:32.906294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-f59d42ae-40cb-4a90-843d-4c6476a4c59f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:32.907490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f59d42ae-40cb-4a90-843d-4c6476a4c59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:32.915576 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1139/2286] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:32.920747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8705543-3ad9-4dfd-ad91-f24eb991436e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:32.931521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8705543-3ad9-4dfd-ad91-f24eb991436e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:32.936217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e156f-5ac9-4fce-8e25-49d42d203795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:32.936850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e156f-5ac9-4fce-8e25-49d42d203795 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:32.937140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e156f-5ac9-4fce-8e25-49d42d203795 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:32.937412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e156f-5ac9-4fce-8e25-49d42d203795 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:32.938054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e156f-5ac9-4fce-8e25-49d42d203795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:54:32.938356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e156f-5ac9-4fce-8e25-49d42d203795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:32.939946 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e156f-5ac9-4fce-8e25-49d42d203795 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:32.951873 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1148/2287] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:54:32.956526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d42ae-40cb-4a90-843d-4c6476a4c59f None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:32.966771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d42ae-40cb-4a90-843d-4c6476a4c59f None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:32.971415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b8f3e-ebcd-4b1b-8c41-dc317ead8c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:32.972003 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b8f3e-ebcd-4b1b-8c41-dc317ead8c86 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:32.972333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b8f3e-ebcd-4b1b-8c41-dc317ead8c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:32.973010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b8f3e-ebcd-4b1b-8c41-dc317ead8c86 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:32.973752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b8f3e-ebcd-4b1b-8c41-dc317ead8c86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:32.975756 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b8f3e-ebcd-4b1b-8c41-dc317ead8c86 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:33.114367 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1140/2288] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:54:32 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:54:33.125028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e156f-5ac9-4fce-8e25-49d42d203795 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:33.135171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e156f-5ac9-4fce-8e25-49d42d203795 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:33.193244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62473dbf-2238-45e0-80ee-c82958799ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:33.193919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62473dbf-2238-45e0-80ee-c82958799ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:33.194241 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62473dbf-2238-45e0-80ee-c82958799ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:33.194564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62473dbf-2238-45e0-80ee-c82958799ff0 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:33.195231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62473dbf-2238-45e0-80ee-c82958799ff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:33.197397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62473dbf-2238-45e0-80ee-c82958799ff0 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:33.290673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1149/2289] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:54:33 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:54:33.294772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b8f3e-ebcd-4b1b-8c41-dc317ead8c86 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:33.305596 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b8f3e-ebcd-4b1b-8c41-dc317ead8c86 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:33.310317 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e4b4e-c36d-49e1-9279-3b07ba2cadd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:33.310919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e4b4e-c36d-49e1-9279-3b07ba2cadd6 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:33.311263 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e4b4e-c36d-49e1-9279-3b07ba2cadd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:33.311545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e4b4e-c36d-49e1-9279-3b07ba2cadd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:33.312208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e4b4e-c36d-49e1-9279-3b07ba2cadd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:33.313869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e4b4e-c36d-49e1-9279-3b07ba2cadd6 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:33.324457 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1141/2290] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:54:33.328889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62473dbf-2238-45e0-80ee-c82958799ff0 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:33.339834 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62473dbf-2238-45e0-80ee-c82958799ff0 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:33.345599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504ac20-6c29-4dfc-8c63-d7cac7668851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:33.346189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504ac20-6c29-4dfc-8c63-d7cac7668851 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:33.349054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504ac20-6c29-4dfc-8c63-d7cac7668851 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:33.349054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504ac20-6c29-4dfc-8c63-d7cac7668851 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:33.349054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504ac20-6c29-4dfc-8c63-d7cac7668851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:33.349392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504ac20-6c29-4dfc-8c63-d7cac7668851 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:33.360476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1150/2291] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:54:33.365046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e4b4e-c36d-49e1-9279-3b07ba2cadd6 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:33.375292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e4b4e-c36d-49e1-9279-3b07ba2cadd6 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:33.380686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5352b04-a7f7-485a-a5aa-9e5a6675236f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:33.381323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5352b04-a7f7-485a-a5aa-9e5a6675236f admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:33.381636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5352b04-a7f7-485a-a5aa-9e5a6675236f admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:33.381992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5352b04-a7f7-485a-a5aa-9e5a6675236f admin admin] PATH_INFO: `/v3/projects/663bebdb63664e248d1534f29b36b02b/users/0e56ac0e7dea4c8f9b33db51efbaffe4/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:33.399116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5352b04-a7f7-485a-a5aa-9e5a6675236f admin admin] RBAC: Authorizing `identity:create_grant(project_id=663bebdb63664e248d1534f29b36b02b, user_id=0e56ac0e7dea4c8f9b33db51efbaffe4, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:33.401878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5352b04-a7f7-485a-a5aa-9e5a6675236f admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:33.508682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1142/2292] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:54:33 2022] PUT /identity/v3/projects/663bebdb63664e248d1534f29b36b02b/users/0e56ac0e7dea4c8f9b33db51efbaffe4/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:54:33.518723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bfc5c-0fff-426c-a9e5-c0a4319db331 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:33.519014 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bfc5c-0fff-426c-a9e5-c0a4319db331 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:33.519843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bfc5c-0fff-426c-a9e5-c0a4319db331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:33.549847 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-923bfc5c-0fff-426c-a9e5-c0a4319db331 None None] MFA Rules not processed for user `0e56ac0e7dea4c8f9b33db51efbaffe4`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:33.551201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-923bfc5c-0fff-426c-a9e5-c0a4319db331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:33.633912 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1151/2293] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:33.646647 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5352b04-a7f7-485a-a5aa-9e5a6675236f admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:33.648688 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c5352b04-a7f7-485a-a5aa-9e5a6675236f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:33.711834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5352b04-a7f7-485a-a5aa-9e5a6675236f admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:33.715457 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3dc2f-b7b8-4119-bb07-bc131cb2e3f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:33.716063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3dc2f-b7b8-4119-bb07-bc131cb2e3f4 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:33.716340 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3dc2f-b7b8-4119-bb07-bc131cb2e3f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:33.716721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3dc2f-b7b8-4119-bb07-bc131cb2e3f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:33.726057 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3dc2f-b7b8-4119-bb07-bc131cb2e3f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:33.728256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3dc2f-b7b8-4119-bb07-bc131cb2e3f4 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:33.738400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1143/2294] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:54:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:54:35.751987 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bfc5c-0fff-426c-a9e5-c0a4319db331 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:35.754111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-923bfc5c-0fff-426c-a9e5-c0a4319db331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:35.797860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:35.822183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bfc5c-0fff-426c-a9e5-c0a4319db331 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:35.826714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3993f-540a-4a82-8ccf-4133b13e9eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:35.827350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3993f-540a-4a82-8ccf-4133b13e9eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:35.827884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3993f-540a-4a82-8ccf-4133b13e9eba admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:35.828160 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3993f-540a-4a82-8ccf-4133b13e9eba admin admin] PATH_INFO: `/v3/users/c8ad14730db741c9bf44fd31a152678b` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:35.829294 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3993f-540a-4a82-8ccf-4133b13e9eba admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8ad14730db741c9bf44fd31a152678b)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:35.831225 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3993f-540a-4a82-8ccf-4133b13e9eba admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:36.012258 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a6c3993f-540a-4a82-8ccf-4133b13e9eba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8ad14730db741c9bf44fd31a152678b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:36.022103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a6c3993f-540a-4a82-8ccf-4133b13e9eba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8ad14730db741c9bf44fd31a152678b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:36.135716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a6c3993f-540a-4a82-8ccf-4133b13e9eba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8ad14730db741c9bf44fd31a152678b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:36.136492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a6c3993f-540a-4a82-8ccf-4133b13e9eba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8ad14730db741c9bf44fd31a152678b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:36.151515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-a6c3993f-540a-4a82-8ccf-4133b13e9eba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8ad14730db741c9bf44fd31a152678b'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:36.157443 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1152/2295] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:54:35 2022] DELETE /identity/v3/users/c8ad14730db741c9bf44fd31a152678b => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:36.167082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a8fa0-1287-4f25-a13e-fb4aaf898957 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:36.167082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a8fa0-1287-4f25-a13e-fb4aaf898957 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:36.167082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a8fa0-1287-4f25-a13e-fb4aaf898957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:36.188250 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-be3a8fa0-1287-4f25-a13e-fb4aaf898957 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:36.189599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-be3a8fa0-1287-4f25-a13e-fb4aaf898957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:36.256804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1144/2296] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:36.266864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3993f-540a-4a82-8ccf-4133b13e9eba admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:36.268256 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a6c3993f-540a-4a82-8ccf-4133b13e9eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:36.334935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3993f-540a-4a82-8ccf-4133b13e9eba admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:36.338893 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201d0a7-f16f-485b-9a1f-8700563fd0a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:36.339569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201d0a7-f16f-485b-9a1f-8700563fd0a5 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:36.339852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201d0a7-f16f-485b-9a1f-8700563fd0a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:36.340125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201d0a7-f16f-485b-9a1f-8700563fd0a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:36.350332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201d0a7-f16f-485b-9a1f-8700563fd0a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:36.351826 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201d0a7-f16f-485b-9a1f-8700563fd0a5 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:36.362164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1153/2297] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:54:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:54:36.824794 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a8fa0-1287-4f25-a13e-fb4aaf898957 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:36.825132 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-be3a8fa0-1287-4f25-a13e-fb4aaf898957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:36.850229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a8fa0-1287-4f25-a13e-fb4aaf898957 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:36.855557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:36.856537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:36.856944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:36.857324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca admin admin] PATH_INFO: `/v3/projects/917f19638f1e4c96b6ecf30de3faeca4` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:36.858954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=917f19638f1e4c96b6ecf30de3faeca4)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:36.861098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:37.023995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:37.036818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '917f19638f1e4c96b6ecf30de3faeca4', 'request_id': 'req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:37.043215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '917f19638f1e4c96b6ecf30de3faeca4', 'request_id': 'req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:37.118246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '917f19638f1e4c96b6ecf30de3faeca4', 'request_id': 'req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:37.131965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca admin admin] The token cache is being invalidate because project 917f19638f1e4c96b6ecf30de3faeca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:54:37.132362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:37.133110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-6f2e8dee-23f5-4ddc-bf60-f1a7ffc8f2ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:37.140338 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c2870-a1a1-4979-a5c9-de03e63cff99 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:37.140942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c2870-a1a1-4979-a5c9-de03e63cff99 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:37.141077 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c2870-a1a1-4979-a5c9-de03e63cff99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:37.144077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1145/2298] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:54:36 2022] DELETE /identity/v3/projects/917f19638f1e4c96b6ecf30de3faeca4 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:37.163436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-d01c2870-a1a1-4979-a5c9-de03e63cff99 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:37.164819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d01c2870-a1a1-4979-a5c9-de03e63cff99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:37.266334 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:37.293026 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1154/2299] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:37.300712 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea537d52-590a-435e-a699-625bae3573c6 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:37.302016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea537d52-590a-435e-a699-625bae3573c6 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:37.302311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea537d52-590a-435e-a699-625bae3573c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:37.326646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-ea537d52-590a-435e-a699-625bae3573c6 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:37.328476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ea537d52-590a-435e-a699-625bae3573c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:37.337541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1146/2300] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:54:37.341943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c2870-a1a1-4979-a5c9-de03e63cff99 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:37.367743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c2870-a1a1-4979-a5c9-de03e63cff99 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:37.372830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a6196-5799-466a-b67a-d42c13c677f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:37.373515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a6196-5799-466a-b67a-d42c13c677f0 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:37.373852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a6196-5799-466a-b67a-d42c13c677f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:37.374139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a6196-5799-466a-b67a-d42c13c677f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:37.374802 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a6196-5799-466a-b67a-d42c13c677f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:54:37.375123 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a6196-5799-466a-b67a-d42c13c677f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:37.376757 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a6196-5799-466a-b67a-d42c13c677f0 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:37.391293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1155/2301] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:54:37.395121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea537d52-590a-435e-a699-625bae3573c6 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:37.409416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea537d52-590a-435e-a699-625bae3573c6 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:37.415167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182c7d8-5c5f-446a-8bd8-39e2e816793e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:37.415852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182c7d8-5c5f-446a-8bd8-39e2e816793e admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:37.416214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182c7d8-5c5f-446a-8bd8-39e2e816793e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:37.416568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182c7d8-5c5f-446a-8bd8-39e2e816793e admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:37.417441 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b182c7d8-5c5f-446a-8bd8-39e2e816793e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:37.419675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b182c7d8-5c5f-446a-8bd8-39e2e816793e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:37.514343 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1147/2302] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:54:37 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:54:37.518700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a6196-5799-466a-b67a-d42c13c677f0 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:37.531460 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a6196-5799-466a-b67a-d42c13c677f0 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:37.595783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a3751-094b-493f-8639-2ca88dd88dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:37.596339 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a3751-094b-493f-8639-2ca88dd88dec admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:37.596623 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a3751-094b-493f-8639-2ca88dd88dec admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:37.596923 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a3751-094b-493f-8639-2ca88dd88dec admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:37.598373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a3751-094b-493f-8639-2ca88dd88dec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:37.600276 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a3751-094b-493f-8639-2ca88dd88dec admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:37.700120 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1156/2303] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:54:37 2022] POST /identity/v3/users => generated 287 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:54:37.701422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182c7d8-5c5f-446a-8bd8-39e2e816793e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:37.713919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182c7d8-5c5f-446a-8bd8-39e2e816793e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:37.718702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8302b-b4ed-4536-a876-0eaa15d8ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:37.719320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8302b-b4ed-4536-a876-0eaa15d8ff11 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:37.719611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8302b-b4ed-4536-a876-0eaa15d8ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:37.719892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8302b-b4ed-4536-a876-0eaa15d8ff11 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:37.720540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8302b-b4ed-4536-a876-0eaa15d8ff11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:37.722329 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8302b-b4ed-4536-a876-0eaa15d8ff11 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:37.732869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1148/2304] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:54:37.736957 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a3751-094b-493f-8639-2ca88dd88dec admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:37.747087 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a3751-094b-493f-8639-2ca88dd88dec admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:37.752333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758d362-b083-493f-9a37-efce8c2c7c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:37.752991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758d362-b083-493f-9a37-efce8c2c7c71 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:37.756482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758d362-b083-493f-9a37-efce8c2c7c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:37.756482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758d362-b083-493f-9a37-efce8c2c7c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:37.756482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758d362-b083-493f-9a37-efce8c2c7c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:37.756753 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758d362-b083-493f-9a37-efce8c2c7c71 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:37.769798 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1157/2305] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:54:37.773577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8302b-b4ed-4536-a876-0eaa15d8ff11 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:37.784612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8302b-b4ed-4536-a876-0eaa15d8ff11 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:37.791413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131d2e3-d002-4935-bc92-2337f349d8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:37.792100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131d2e3-d002-4935-bc92-2337f349d8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:37.792404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131d2e3-d002-4935-bc92-2337f349d8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:37.792706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131d2e3-d002-4935-bc92-2337f349d8b2 admin admin] PATH_INFO: `/v3/projects/587de61373174a6e806f4241388a0f97/users/03611f037b5a4371ac06d63f0cf7cda3/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:37.811215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131d2e3-d002-4935-bc92-2337f349d8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=587de61373174a6e806f4241388a0f97, user_id=03611f037b5a4371ac06d63f0cf7cda3, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:37.814103 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131d2e3-d002-4935-bc92-2337f349d8b2 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:37.933967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1149/2306] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:54:37 2022] PUT /identity/v3/projects/587de61373174a6e806f4241388a0f97/users/03611f037b5a4371ac06d63f0cf7cda3/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:54:37.949510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29ec88-3f82-4c9b-8183-d07b562c825e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:37.951139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29ec88-3f82-4c9b-8183-d07b562c825e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:37.951139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29ec88-3f82-4c9b-8183-d07b562c825e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:37.986433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0d29ec88-3f82-4c9b-8183-d07b562c825e None None] MFA Rules not processed for user `03611f037b5a4371ac06d63f0cf7cda3`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:37.987725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0d29ec88-3f82-4c9b-8183-d07b562c825e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:38.088180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1158/2307] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:38.095820 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131d2e3-d002-4935-bc92-2337f349d8b2 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:38.098222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0131d2e3-d002-4935-bc92-2337f349d8b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:38.166804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131d2e3-d002-4935-bc92-2337f349d8b2 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:38.179215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41498c74-c31f-40da-ad49-c3852e846d39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:38.179215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41498c74-c31f-40da-ad49-c3852e846d39 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:38.179215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41498c74-c31f-40da-ad49-c3852e846d39 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:38.179215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41498c74-c31f-40da-ad49-c3852e846d39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:38.188110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41498c74-c31f-40da-ad49-c3852e846d39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:38.189620 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41498c74-c31f-40da-ad49-c3852e846d39 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:38.199592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1150/2308] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:54:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:54:42.675189 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29ec88-3f82-4c9b-8183-d07b562c825e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:42.794708 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:42.800976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29ec88-3f82-4c9b-8183-d07b562c825e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:42.806094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425065a-fd02-4654-b0d2-a4ed9d21acc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:42.806823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425065a-fd02-4654-b0d2-a4ed9d21acc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:42.807108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425065a-fd02-4654-b0d2-a4ed9d21acc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:42.807387 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425065a-fd02-4654-b0d2-a4ed9d21acc2 admin admin] PATH_INFO: `/v3/users/03611f037b5a4371ac06d63f0cf7cda3` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:42.808555 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425065a-fd02-4654-b0d2-a4ed9d21acc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03611f037b5a4371ac06d63f0cf7cda3)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:42.810553 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425065a-fd02-4654-b0d2-a4ed9d21acc2 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:43.095301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0425065a-fd02-4654-b0d2-a4ed9d21acc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03611f037b5a4371ac06d63f0cf7cda3'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:43.103254 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0425065a-fd02-4654-b0d2-a4ed9d21acc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03611f037b5a4371ac06d63f0cf7cda3'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:43.181383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0425065a-fd02-4654-b0d2-a4ed9d21acc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03611f037b5a4371ac06d63f0cf7cda3'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:43.182258 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0425065a-fd02-4654-b0d2-a4ed9d21acc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03611f037b5a4371ac06d63f0cf7cda3'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:43.195945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0425065a-fd02-4654-b0d2-a4ed9d21acc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03611f037b5a4371ac06d63f0cf7cda3'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:43.201997 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1159/2309] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:54:42 2022] DELETE /identity/v3/users/03611f037b5a4371ac06d63f0cf7cda3 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:43.207507 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919e716-764f-4468-9185-7985e195a180 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:43.208087 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919e716-764f-4468-9185-7985e195a180 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:43.208087 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919e716-764f-4468-9185-7985e195a180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:43.227802 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-6919e716-764f-4468-9185-7985e195a180 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:43.228961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6919e716-764f-4468-9185-7985e195a180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:43.280361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:43.288724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1151/2310] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:43.300538 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425065a-fd02-4654-b0d2-a4ed9d21acc2 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:43.302173 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0425065a-fd02-4654-b0d2-a4ed9d21acc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:43.367228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425065a-fd02-4654-b0d2-a4ed9d21acc2 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:43.371101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54586847-37b2-4aca-95d7-9a30fcc86328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:43.371788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54586847-37b2-4aca-95d7-9a30fcc86328 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:43.372072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54586847-37b2-4aca-95d7-9a30fcc86328 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:43.372342 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54586847-37b2-4aca-95d7-9a30fcc86328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:43.384847 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54586847-37b2-4aca-95d7-9a30fcc86328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:43.387338 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54586847-37b2-4aca-95d7-9a30fcc86328 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:43.399086 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1160/2311] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:54:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:54:43.914044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919e716-764f-4468-9185-7985e195a180 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:43.916451 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6919e716-764f-4468-9185-7985e195a180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:43.936893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919e716-764f-4468-9185-7985e195a180 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:43.947457 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b040b71-969a-442e-871b-bf3aa0b4c7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:43.948574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b040b71-969a-442e-871b-bf3aa0b4c7f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:43.949893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b040b71-969a-442e-871b-bf3aa0b4c7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:43.951399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b040b71-969a-442e-871b-bf3aa0b4c7f9 admin admin] PATH_INFO: `/v3/projects/587de61373174a6e806f4241388a0f97` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:43.953328 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b040b71-969a-442e-871b-bf3aa0b4c7f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=587de61373174a6e806f4241388a0f97)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:43.955267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b040b71-969a-442e-871b-bf3aa0b4c7f9 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:44.135526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4b040b71-969a-442e-871b-bf3aa0b4c7f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '587de61373174a6e806f4241388a0f97', 'request_id': 'req-4b040b71-969a-442e-871b-bf3aa0b4c7f9'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:44.141934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4b040b71-969a-442e-871b-bf3aa0b4c7f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '587de61373174a6e806f4241388a0f97', 'request_id': 'req-4b040b71-969a-442e-871b-bf3aa0b4c7f9'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:44.226258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4b040b71-969a-442e-871b-bf3aa0b4c7f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '587de61373174a6e806f4241388a0f97', 'request_id': 'req-4b040b71-969a-442e-871b-bf3aa0b4c7f9'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:44.237152 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4b040b71-969a-442e-871b-bf3aa0b4c7f9 admin admin] The token cache is being invalidate because project 587de61373174a6e806f4241388a0f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:54:44.237582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4b040b71-969a-442e-871b-bf3aa0b4c7f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:44.238213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4b040b71-969a-442e-871b-bf3aa0b4c7f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:44.239926 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1152/2312] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:54:43 2022] DELETE /identity/v3/projects/587de61373174a6e806f4241388a0f97 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:44.244763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832b7b5-12a6-478f-9e78-f88db69885f5 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:44.245231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832b7b5-12a6-478f-9e78-f88db69885f5 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:44.245327 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832b7b5-12a6-478f-9e78-f88db69885f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:44.264720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-f832b7b5-12a6-478f-9e78-f88db69885f5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:44.265963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f832b7b5-12a6-478f-9e78-f88db69885f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:44.326536 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1161/2313] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:44.338067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12f833-89fb-4f5f-bf3a-d642f5ae9058 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:44.338383 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12f833-89fb-4f5f-bf3a-d642f5ae9058 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:44.338657 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12f833-89fb-4f5f-bf3a-d642f5ae9058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:44.360154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-0a12f833-89fb-4f5f-bf3a-d642f5ae9058 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:44.361432 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0a12f833-89fb-4f5f-bf3a-d642f5ae9058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:44.369675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1153/2314] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:44.374779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f832b7b5-12a6-478f-9e78-f88db69885f5 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:44.384551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f832b7b5-12a6-478f-9e78-f88db69885f5 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:44.389213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948cd4a-33a8-4bbc-a07a-52bac58fdf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:44.389807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948cd4a-33a8-4bbc-a07a-52bac58fdf0c admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:44.390093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948cd4a-33a8-4bbc-a07a-52bac58fdf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:44.390370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948cd4a-33a8-4bbc-a07a-52bac58fdf0c admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:44.391015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948cd4a-33a8-4bbc-a07a-52bac58fdf0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:54:44.391327 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948cd4a-33a8-4bbc-a07a-52bac58fdf0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:44.392961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948cd4a-33a8-4bbc-a07a-52bac58fdf0c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:44.405006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1162/2315] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:54:44.409979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12f833-89fb-4f5f-bf3a-d642f5ae9058 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:44.419088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12f833-89fb-4f5f-bf3a-d642f5ae9058 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:44.423764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31aa14-2c89-47f3-9c01-8caaf19fda8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:44.424355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31aa14-2c89-47f3-9c01-8caaf19fda8a admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:44.424658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31aa14-2c89-47f3-9c01-8caaf19fda8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:44.425007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31aa14-2c89-47f3-9c01-8caaf19fda8a admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:44.425656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31aa14-2c89-47f3-9c01-8caaf19fda8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:44.427558 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31aa14-2c89-47f3-9c01-8caaf19fda8a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:44.548188 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1154/2316] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:54:44 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:54:44.552858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948cd4a-33a8-4bbc-a07a-52bac58fdf0c admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:44.559328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:44.564171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948cd4a-33a8-4bbc-a07a-52bac58fdf0c admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:44.622996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6413b-19d4-40c2-a493-c4641d191c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:44.623523 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6413b-19d4-40c2-a493-c4641d191c78 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:44.623804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6413b-19d4-40c2-a493-c4641d191c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:44.624125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6413b-19d4-40c2-a493-c4641d191c78 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:44.624985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d6413b-19d4-40c2-a493-c4641d191c78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:44.626918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d6413b-19d4-40c2-a493-c4641d191c78 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:44.721535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1163/2317] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:54:44 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:54:44.722962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31aa14-2c89-47f3-9c01-8caaf19fda8a admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:44.733805 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31aa14-2c89-47f3-9c01-8caaf19fda8a admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:44.738779 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21747cae-60db-4089-8a98-974813807fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:44.739692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21747cae-60db-4089-8a98-974813807fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:44.740118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21747cae-60db-4089-8a98-974813807fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:44.740422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21747cae-60db-4089-8a98-974813807fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:44.741087 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21747cae-60db-4089-8a98-974813807fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:44.742739 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21747cae-60db-4089-8a98-974813807fd2 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:44.754576 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1155/2318] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:54:44.758781 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6413b-19d4-40c2-a493-c4641d191c78 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:44.769540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6413b-19d4-40c2-a493-c4641d191c78 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:44.774474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4465bc-1b24-4a82-9925-bbc6047599a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:44.775076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4465bc-1b24-4a82-9925-bbc6047599a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:44.775358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4465bc-1b24-4a82-9925-bbc6047599a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:44.775642 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4465bc-1b24-4a82-9925-bbc6047599a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:44.776348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4465bc-1b24-4a82-9925-bbc6047599a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:44.777965 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4465bc-1b24-4a82-9925-bbc6047599a8 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:44.789666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1164/2319] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:54:44.793956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21747cae-60db-4089-8a98-974813807fd2 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:44.807917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21747cae-60db-4089-8a98-974813807fd2 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:44.812711 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23dc3a-c75a-4dd8-8e83-4012669e9051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:44.813417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23dc3a-c75a-4dd8-8e83-4012669e9051 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:44.813826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23dc3a-c75a-4dd8-8e83-4012669e9051 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:44.814166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23dc3a-c75a-4dd8-8e83-4012669e9051 admin admin] PATH_INFO: `/v3/projects/d5879442f82a40ca907205d147cea5df/users/4bedc4336c374971842b9d377decb462/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:44.829874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23dc3a-c75a-4dd8-8e83-4012669e9051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5879442f82a40ca907205d147cea5df, user_id=4bedc4336c374971842b9d377decb462, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:44.832609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23dc3a-c75a-4dd8-8e83-4012669e9051 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:44.993338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1156/2320] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:54:44 2022] PUT /identity/v3/projects/d5879442f82a40ca907205d147cea5df/users/4bedc4336c374971842b9d377decb462/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:45.004876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db435d76-b2bb-4a6f-93bd-65ac8ec8f572 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:45.005207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db435d76-b2bb-4a6f-93bd-65ac8ec8f572 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:45.005675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db435d76-b2bb-4a6f-93bd-65ac8ec8f572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:45.037783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-db435d76-b2bb-4a6f-93bd-65ac8ec8f572 None None] MFA Rules not processed for user `4bedc4336c374971842b9d377decb462`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:45.039014 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-db435d76-b2bb-4a6f-93bd-65ac8ec8f572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:45.107820 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1165/2321] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:45.120628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23dc3a-c75a-4dd8-8e83-4012669e9051 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:45.122632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ac23dc3a-c75a-4dd8-8e83-4012669e9051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:45.193452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23dc3a-c75a-4dd8-8e83-4012669e9051 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:45.200796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd2e40-fb9d-436f-99df-7a9ffb9910c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:45.200796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd2e40-fb9d-436f-99df-7a9ffb9910c6 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:45.200796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd2e40-fb9d-436f-99df-7a9ffb9910c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:45.200796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd2e40-fb9d-436f-99df-7a9ffb9910c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:45.212159 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd2e40-fb9d-436f-99df-7a9ffb9910c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:45.213663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd2e40-fb9d-436f-99df-7a9ffb9910c6 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:45.223124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1157/2322] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:54:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:54:46.235172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db435d76-b2bb-4a6f-93bd-65ac8ec8f572 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:46.236776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-db435d76-b2bb-4a6f-93bd-65ac8ec8f572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:46.305475 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db435d76-b2bb-4a6f-93bd-65ac8ec8f572 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:46.310312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3272344-9c33-4fe7-a70c-a4830e545373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:46.311136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3272344-9c33-4fe7-a70c-a4830e545373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:46.311517 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3272344-9c33-4fe7-a70c-a4830e545373 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:46.311865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3272344-9c33-4fe7-a70c-a4830e545373 admin admin] PATH_INFO: `/v3/users/0e56ac0e7dea4c8f9b33db51efbaffe4` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:46.313099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3272344-9c33-4fe7-a70c-a4830e545373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e56ac0e7dea4c8f9b33db51efbaffe4)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:46.315154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3272344-9c33-4fe7-a70c-a4830e545373 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:46.330698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:46.518965 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b3272344-9c33-4fe7-a70c-a4830e545373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e56ac0e7dea4c8f9b33db51efbaffe4'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:46.525687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b3272344-9c33-4fe7-a70c-a4830e545373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e56ac0e7dea4c8f9b33db51efbaffe4'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:46.603411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b3272344-9c33-4fe7-a70c-a4830e545373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e56ac0e7dea4c8f9b33db51efbaffe4'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:46.603984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b3272344-9c33-4fe7-a70c-a4830e545373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e56ac0e7dea4c8f9b33db51efbaffe4'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:46.617595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b3272344-9c33-4fe7-a70c-a4830e545373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e56ac0e7dea4c8f9b33db51efbaffe4'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:46.623462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1166/2323] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:54:46 2022] DELETE /identity/v3/users/0e56ac0e7dea4c8f9b33db51efbaffe4 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:46.627866 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ecf5e-ecab-412d-9e8f-aabae208e58e None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:46.628985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ecf5e-ecab-412d-9e8f-aabae208e58e None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:46.629158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ecf5e-ecab-412d-9e8f-aabae208e58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:46.647999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-0d5ecf5e-ecab-412d-9e8f-aabae208e58e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:46.649223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0d5ecf5e-ecab-412d-9e8f-aabae208e58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:46.665622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:46.707571 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1158/2324] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:46.717465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3272344-9c33-4fe7-a70c-a4830e545373 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:46.718962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b3272344-9c33-4fe7-a70c-a4830e545373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:46.778219 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3272344-9c33-4fe7-a70c-a4830e545373 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:46.781930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb912cc-9972-4113-b779-3877cf0d0ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:46.782525 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb912cc-9972-4113-b779-3877cf0d0ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:46.782809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb912cc-9972-4113-b779-3877cf0d0ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:46.783083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb912cc-9972-4113-b779-3877cf0d0ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:46.792806 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb912cc-9972-4113-b779-3877cf0d0ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:46.794320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb912cc-9972-4113-b779-3877cf0d0ec6 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:46.804677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1167/2325] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:54:47.276109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ecf5e-ecab-412d-9e8f-aabae208e58e None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:54:47.276567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0d5ecf5e-ecab-412d-9e8f-aabae208e58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:47.290794 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ecf5e-ecab-412d-9e8f-aabae208e58e None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:54:47.296031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:54:47.296721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:47.297020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:47.297375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] PATH_INFO: `/v3/projects/663bebdb63664e248d1534f29b36b02b` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:47.298442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=663bebdb63664e248d1534f29b36b02b)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:54:47.300801 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:54:47.405933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '663bebdb63664e248d1534f29b36b02b', 'request_id': 'req-1aee8501-0fad-40e7-aec3-1a949b49fbaa'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:47.412828 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '663bebdb63664e248d1534f29b36b02b', 'request_id': 'req-1aee8501-0fad-40e7-aec3-1a949b49fbaa'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:47.460887 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '663bebdb63664e248d1534f29b36b02b', 'request_id': 'req-1aee8501-0fad-40e7-aec3-1a949b49fbaa'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:47.472444 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] The token cache is being invalidate because project 663bebdb63664e248d1534f29b36b02b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:54:47.472769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:47.473415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:54:47.474928 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1159/2326] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:54:47 2022] DELETE /identity/v3/projects/663bebdb63664e248d1534f29b36b02b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:54:48.134973 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d97f49-91a9-4b78-880a-2ad12565eeaa None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:54:48.135499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d97f49-91a9-4b78-880a-2ad12565eeaa None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:54:48.135499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d97f49-91a9-4b78-880a-2ad12565eeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:54:48.155789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-39d97f49-91a9-4b78-880a-2ad12565eeaa None None] MFA Rules not processed for user `9a4f3bfb16c6418cbdc1b6e8d4457a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:54:48.157230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-39d97f49-91a9-4b78-880a-2ad12565eeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:54:48.215242 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1168/2327] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:54:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:54:57.468763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:58.208204 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=72,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:00.460947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:00.462455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:00.535599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee8501-0fad-40e7-aec3-1a949b49fbaa admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:00.540739 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613ba50-01ff-47c0-a312-9c6df141fd84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:00.541352 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613ba50-01ff-47c0-a312-9c6df141fd84 service placement] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:00.541634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613ba50-01ff-47c0-a312-9c6df141fd84 service placement] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:00.541912 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613ba50-01ff-47c0-a312-9c6df141fd84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:00.543657 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f613ba50-01ff-47c0-a312-9c6df141fd84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:00.555992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613ba50-01ff-47c0-a312-9c6df141fd84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:00.557642 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613ba50-01ff-47c0-a312-9c6df141fd84 service placement] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:00.568171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1160/2328] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:01.162280 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d97f49-91a9-4b78-880a-2ad12565eeaa None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:01.164522 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-39d97f49-91a9-4b78-880a-2ad12565eeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:01.224088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d97f49-91a9-4b78-880a-2ad12565eeaa None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:01.228691 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c69e30-bd51-4f84-b370-3d3c9198241f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:01.229307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c69e30-bd51-4f84-b370-3d3c9198241f service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:01.229587 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c69e30-bd51-4f84-b370-3d3c9198241f service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:01.229858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c69e30-bd51-4f84-b370-3d3c9198241f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:01.231634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f4c69e30-bd51-4f84-b370-3d3c9198241f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:01.297250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c69e30-bd51-4f84-b370-3d3c9198241f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:01.298686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c69e30-bd51-4f84-b370-3d3c9198241f service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:01.308790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1169/2329] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:10.563458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=32 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:11.304942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=50 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:11.895781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613ba50-01ff-47c0-a312-9c6df141fd84 service placement] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:11.906361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613ba50-01ff-47c0-a312-9c6df141fd84 service placement] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:11.910480 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41551b05-2f5f-4c61-98ad-a4e70c5e1eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:11.911233 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551b05-2f5f-4c61-98ad-a4e70c5e1eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:11.911512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551b05-2f5f-4c61-98ad-a4e70c5e1eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:11.911783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551b05-2f5f-4c61-98ad-a4e70c5e1eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:11.913751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-41551b05-2f5f-4c61-98ad-a4e70c5e1eb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:11.979505 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41551b05-2f5f-4c61-98ad-a4e70c5e1eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:11.980977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41551b05-2f5f-4c61-98ad-a4e70c5e1eb3 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:11.991024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1161/2330] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:55:21.986229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=30 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:30.151108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c69e30-bd51-4f84-b370-3d3c9198241f service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:30.152580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f4c69e30-bd51-4f84-b370-3d3c9198241f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:30.218478 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c69e30-bd51-4f84-b370-3d3c9198241f service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:30.223834 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58f222-5bc7-4d89-8421-b71349d4e223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:30.224468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58f222-5bc7-4d89-8421-b71349d4e223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:30.224780 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58f222-5bc7-4d89-8421-b71349d4e223 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:30.225053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58f222-5bc7-4d89-8421-b71349d4e223 admin admin] PATH_INFO: `/v3/users/4bedc4336c374971842b9d377decb462` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:30.226159 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58f222-5bc7-4d89-8421-b71349d4e223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bedc4336c374971842b9d377decb462)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:30.228124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58f222-5bc7-4d89-8421-b71349d4e223 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:30.467111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2c58f222-5bc7-4d89-8421-b71349d4e223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bedc4336c374971842b9d377decb462'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.474548 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2c58f222-5bc7-4d89-8421-b71349d4e223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bedc4336c374971842b9d377decb462'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.597577 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2c58f222-5bc7-4d89-8421-b71349d4e223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bedc4336c374971842b9d377decb462'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.598257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2c58f222-5bc7-4d89-8421-b71349d4e223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bedc4336c374971842b9d377decb462'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.610731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2c58f222-5bc7-4d89-8421-b71349d4e223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bedc4336c374971842b9d377decb462'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.616985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1170/2331] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:30 2022] DELETE /identity/v3/users/4bedc4336c374971842b9d377decb462 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:30.621352 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c50f3-fea1-49b8-bde8-74b725a40bb5 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:30.621866 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c50f3-fea1-49b8-bde8-74b725a40bb5 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:30.621979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c50f3-fea1-49b8-bde8-74b725a40bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:30.641613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-113c50f3-fea1-49b8-bde8-74b725a40bb5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:30.642822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-113c50f3-fea1-49b8-bde8-74b725a40bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:30.706903 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1162/2332] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:30.722170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58f222-5bc7-4d89-8421-b71349d4e223 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:30.722170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2c58f222-5bc7-4d89-8421-b71349d4e223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:30.776100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58f222-5bc7-4d89-8421-b71349d4e223 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:30.790956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe96ac-e5c1-42c3-8636-71fe05b62a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:30.790956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe96ac-e5c1-42c3-8636-71fe05b62a43 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:30.790956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe96ac-e5c1-42c3-8636-71fe05b62a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:30.790956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe96ac-e5c1-42c3-8636-71fe05b62a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:30.791362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe96ac-e5c1-42c3-8636-71fe05b62a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:30.791721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe96ac-e5c1-42c3-8636-71fe05b62a43 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:30.801693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1171/2333] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:31.317057 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c50f3-fea1-49b8-bde8-74b725a40bb5 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:31.317057 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-113c50f3-fea1-49b8-bde8-74b725a40bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:31.320568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c50f3-fea1-49b8-bde8-74b725a40bb5 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:31.325425 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768723e-d0e5-4506-822e-cc1b6927232e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:31.326061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768723e-d0e5-4506-822e-cc1b6927232e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:31.326603 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768723e-d0e5-4506-822e-cc1b6927232e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:31.326879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768723e-d0e5-4506-822e-cc1b6927232e admin admin] PATH_INFO: `/v3/projects/d5879442f82a40ca907205d147cea5df` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:31.328011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768723e-d0e5-4506-822e-cc1b6927232e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5879442f82a40ca907205d147cea5df)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:31.329962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768723e-d0e5-4506-822e-cc1b6927232e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:31.489386 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a768723e-d0e5-4506-822e-cc1b6927232e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5879442f82a40ca907205d147cea5df', 'request_id': 'req-a768723e-d0e5-4506-822e-cc1b6927232e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:31.495434 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a768723e-d0e5-4506-822e-cc1b6927232e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5879442f82a40ca907205d147cea5df', 'request_id': 'req-a768723e-d0e5-4506-822e-cc1b6927232e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:31.568697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a768723e-d0e5-4506-822e-cc1b6927232e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5879442f82a40ca907205d147cea5df', 'request_id': 'req-a768723e-d0e5-4506-822e-cc1b6927232e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:31.579947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a768723e-d0e5-4506-822e-cc1b6927232e admin admin] The token cache is being invalidate because project d5879442f82a40ca907205d147cea5df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:55:31.580238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a768723e-d0e5-4506-822e-cc1b6927232e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:31.580986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a768723e-d0e5-4506-822e-cc1b6927232e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:31.582486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1163/2334] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:55:31 2022] DELETE /identity/v3/projects/d5879442f82a40ca907205d147cea5df => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:31.587467 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52f15c-b5c8-412e-a44c-c1caf1be5d39 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:31.587947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52f15c-b5c8-412e-a44c-c1caf1be5d39 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:31.588057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52f15c-b5c8-412e-a44c-c1caf1be5d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:31.608592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-4a52f15c-b5c8-412e-a44c-c1caf1be5d39 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:31.609818 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4a52f15c-b5c8-412e-a44c-c1caf1be5d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:31.627023 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:31.668016 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1172/2335] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:31.676925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4da4e5-59a5-4e81-9a5b-a8f118b7a226 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:31.677491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4da4e5-59a5-4e81-9a5b-a8f118b7a226 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:31.677491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4da4e5-59a5-4e81-9a5b-a8f118b7a226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:31.699073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-fa4da4e5-59a5-4e81-9a5b-a8f118b7a226 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:31.700005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fa4da4e5-59a5-4e81-9a5b-a8f118b7a226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:31.709027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1164/2336] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:31.713163 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52f15c-b5c8-412e-a44c-c1caf1be5d39 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:31.723986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52f15c-b5c8-412e-a44c-c1caf1be5d39 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:31.732929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1b24d-501e-4f51-9fa9-707ba4bf97b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:31.732929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1b24d-501e-4f51-9fa9-707ba4bf97b5 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:31.732929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1b24d-501e-4f51-9fa9-707ba4bf97b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:31.732929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1b24d-501e-4f51-9fa9-707ba4bf97b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:31.733360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1b24d-501e-4f51-9fa9-707ba4bf97b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:31.733360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1b24d-501e-4f51-9fa9-707ba4bf97b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:31.734894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1b24d-501e-4f51-9fa9-707ba4bf97b5 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:31.750528 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1173/2337] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:31.752098 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4da4e5-59a5-4e81-9a5b-a8f118b7a226 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:31.762199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4da4e5-59a5-4e81-9a5b-a8f118b7a226 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:31.773266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d3150-ed57-47e0-87a9-b8ecc7744f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:31.773266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d3150-ed57-47e0-87a9-b8ecc7744f45 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:31.773266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d3150-ed57-47e0-87a9-b8ecc7744f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:31.773266 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d3150-ed57-47e0-87a9-b8ecc7744f45 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:31.773640 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d3150-ed57-47e0-87a9-b8ecc7744f45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:31.773640 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d3150-ed57-47e0-87a9-b8ecc7744f45 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:31.868111 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1165/2338] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:31 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:31.872103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1b24d-501e-4f51-9fa9-707ba4bf97b5 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:31.883099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1b24d-501e-4f51-9fa9-707ba4bf97b5 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:31.940096 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eea9c4-4423-42f1-8209-609f4a91c5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:31.941314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eea9c4-4423-42f1-8209-609f4a91c5ee admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:31.941596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eea9c4-4423-42f1-8209-609f4a91c5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:31.941870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eea9c4-4423-42f1-8209-609f4a91c5ee admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:31.942488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eea9c4-4423-42f1-8209-609f4a91c5ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:31.944418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eea9c4-4423-42f1-8209-609f4a91c5ee admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:32.059950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1174/2339] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:31 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:32.064232 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d3150-ed57-47e0-87a9-b8ecc7744f45 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:32.075443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d3150-ed57-47e0-87a9-b8ecc7744f45 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:32.080193 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3a959-b63f-4a0b-a597-6261992418fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:32.080931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3a959-b63f-4a0b-a597-6261992418fa admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:32.086840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3a959-b63f-4a0b-a597-6261992418fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:32.086840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3a959-b63f-4a0b-a597-6261992418fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:32.086840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3a959-b63f-4a0b-a597-6261992418fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:32.086840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3a959-b63f-4a0b-a597-6261992418fa admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:32.093972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1166/2340] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:32.098206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eea9c4-4423-42f1-8209-609f4a91c5ee admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:32.108067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eea9c4-4423-42f1-8209-609f4a91c5ee admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:32.113866 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e797706-c1e7-464d-8ef6-fb0f63b2c629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:32.114470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e797706-c1e7-464d-8ef6-fb0f63b2c629 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:32.114752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e797706-c1e7-464d-8ef6-fb0f63b2c629 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:32.115050 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e797706-c1e7-464d-8ef6-fb0f63b2c629 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:32.115689 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e797706-c1e7-464d-8ef6-fb0f63b2c629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:32.117411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e797706-c1e7-464d-8ef6-fb0f63b2c629 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:32.130645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1175/2341] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:32.134665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3a959-b63f-4a0b-a597-6261992418fa admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:32.145183 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3a959-b63f-4a0b-a597-6261992418fa admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:32.149885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf237d-9c4b-4db9-af9c-1237ea4d2c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:32.155629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf237d-9c4b-4db9-af9c-1237ea4d2c1f admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:32.155629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf237d-9c4b-4db9-af9c-1237ea4d2c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:32.155629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf237d-9c4b-4db9-af9c-1237ea4d2c1f admin admin] PATH_INFO: `/v3/projects/05b468fa5d3046ebb37fbd43102be471/users/ab1a288170374d1a80355c869c78b4ca/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:32.165710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf237d-9c4b-4db9-af9c-1237ea4d2c1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b468fa5d3046ebb37fbd43102be471, user_id=ab1a288170374d1a80355c869c78b4ca, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:32.168372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf237d-9c4b-4db9-af9c-1237ea4d2c1f admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:32.255944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1167/2342] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:32 2022] PUT /identity/v3/projects/05b468fa5d3046ebb37fbd43102be471/users/ab1a288170374d1a80355c869c78b4ca/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:32.261618 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab52c17-6496-4f0d-b587-2a1b0437900b None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:32.261910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab52c17-6496-4f0d-b587-2a1b0437900b None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:32.262188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab52c17-6496-4f0d-b587-2a1b0437900b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:32.281491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-fab52c17-6496-4f0d-b587-2a1b0437900b None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:32.282712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-fab52c17-6496-4f0d-b587-2a1b0437900b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:32.352444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1176/2343] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:32.365440 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf237d-9c4b-4db9-af9c-1237ea4d2c1f admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:32.367686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2baf237d-9c4b-4db9-af9c-1237ea4d2c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:32.398902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:32.435895 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf237d-9c4b-4db9-af9c-1237ea4d2c1f admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:32.439609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085aedec-e848-497a-9add-f8c72949077d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:32.440942 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085aedec-e848-497a-9add-f8c72949077d service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:32.441189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085aedec-e848-497a-9add-f8c72949077d service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:32.441428 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085aedec-e848-497a-9add-f8c72949077d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:32.451095 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085aedec-e848-497a-9add-f8c72949077d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:32.452957 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085aedec-e848-497a-9add-f8c72949077d service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:32.464113 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1168/2344] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:33.786443 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6b2ba-8988-4252-be08-333577ed2856 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:33.786957 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6b2ba-8988-4252-be08-333577ed2856 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:33.787080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6b2ba-8988-4252-be08-333577ed2856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:33.817372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-c7d6b2ba-8988-4252-be08-333577ed2856 None None] MFA Rules not processed for user `ab1a288170374d1a80355c869c78b4ca`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:33.818905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c7d6b2ba-8988-4252-be08-333577ed2856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:33.837199 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:33.885452 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1177/2345] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:55:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:33.896809 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085aedec-e848-497a-9add-f8c72949077d service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:33.898241 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-085aedec-e848-497a-9add-f8c72949077d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:33.954796 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085aedec-e848-497a-9add-f8c72949077d service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:33.958545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12627a8-6dba-432d-b6dd-2fc742a6aaf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:33.959146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12627a8-6dba-432d-b6dd-2fc742a6aaf0 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:33.959427 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12627a8-6dba-432d-b6dd-2fc742a6aaf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:33.959702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12627a8-6dba-432d-b6dd-2fc742a6aaf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:33.967728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12627a8-6dba-432d-b6dd-2fc742a6aaf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:33.969204 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12627a8-6dba-432d-b6dd-2fc742a6aaf0 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:33.978274 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1169/2346] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:35.738405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077f975-5517-42c6-9539-295c57880bdc None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:35.739374 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077f975-5517-42c6-9539-295c57880bdc None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:35.739836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077f975-5517-42c6-9539-295c57880bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:35.768606 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0077f975-5517-42c6-9539-295c57880bdc None None] MFA Rules not processed for user `ab1a288170374d1a80355c869c78b4ca`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:35.770057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0077f975-5517-42c6-9539-295c57880bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:35.779187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1178/2347] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:55:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:40.970871 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b614e-ed69-4462-8720-cdade9e980c4 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:40.971475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b614e-ed69-4462-8720-cdade9e980c4 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:40.971581 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b614e-ed69-4462-8720-cdade9e980c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:40.993196 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-8e6b614e-ed69-4462-8720-cdade9e980c4 None None] MFA Rules not processed for user `aee01beeba5b48c3b6c25489743de40e`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:40.994316 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-8e6b614e-ed69-4462-8720-cdade9e980c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:41.051883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1170/2348] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:55:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:45.760388 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=22 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:51.044169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=68 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:06.514673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc94d5-d1d0-4ae6-8d94-008e4bfcd47e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:06.515188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc94d5-d1d0-4ae6-8d94-008e4bfcd47e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:06.515188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc94d5-d1d0-4ae6-8d94-008e4bfcd47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:06.534617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0bcc94d5-d1d0-4ae6-8d94-008e4bfcd47e None None] MFA Rules not processed for user `e23edadcfe3d4a0b81b13e278682ca2f`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:06.535860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0bcc94d5-d1d0-4ae6-8d94-008e4bfcd47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:06.590947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1179/2349] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:56:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:16.584286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=25 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:49.969868 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f48fc-5847-483f-8df8-4ff224cd07e6 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:49.976188 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f48fc-5847-483f-8df8-4ff224cd07e6 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:49.976188 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f48fc-5847-483f-8df8-4ff224cd07e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:49.990275 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-757f48fc-5847-483f-8df8-4ff224cd07e6 None None] MFA Rules not processed for user `2ec3daa674814619976aead75459e98c`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:49.991476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-757f48fc-5847-483f-8df8-4ff224cd07e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:50.045066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1171/2350] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:56:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:50.086170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc94d5-d1d0-4ae6-8d94-008e4bfcd47e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:50.088169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0bcc94d5-d1d0-4ae6-8d94-008e4bfcd47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:50.147467 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc94d5-d1d0-4ae6-8d94-008e4bfcd47e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:50.151091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13a2ba-9868-4f9e-968a-4c104d7e708e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38de805ba5eb4aa58cf95d2082c2d216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:50.151688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13a2ba-9868-4f9e-968a-4c104d7e708e service glance] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:50.151969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13a2ba-9868-4f9e-968a-4c104d7e708e service glance] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:50.152238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13a2ba-9868-4f9e-968a-4c104d7e708e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:50.153948 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9d13a2ba-9868-4f9e-968a-4c104d7e708e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:50.224630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13a2ba-9868-4f9e-968a-4c104d7e708e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:50.226001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13a2ba-9868-4f9e-968a-4c104d7e708e service glance] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:50.237157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1180/2351] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:56:50.935153 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f48fc-5847-483f-8df8-4ff224cd07e6 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:50.936577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-757f48fc-5847-483f-8df8-4ff224cd07e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:50.949138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f48fc-5847-483f-8df8-4ff224cd07e6 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:50.952988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb9f94-5de4-4fef-8421-90df9e79f6dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:50.953617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb9f94-5de4-4fef-8421-90df9e79f6dd service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:50.953898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb9f94-5de4-4fef-8421-90df9e79f6dd service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:50.954179 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb9f94-5de4-4fef-8421-90df9e79f6dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:50.955995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-94cb9f94-5de4-4fef-8421-90df9e79f6dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:50.969791 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb9f94-5de4-4fef-8421-90df9e79f6dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:50.971293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb9f94-5de4-4fef-8421-90df9e79f6dd service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:50.982460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1172/2352] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:55.121598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13a2ba-9868-4f9e-968a-4c104d7e708e service glance] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:55.122985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9d13a2ba-9868-4f9e-968a-4c104d7e708e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:55.134194 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13a2ba-9868-4f9e-968a-4c104d7e708e service glance] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:55.137804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25eaebd-e8f4-4571-95ec-d290af114554 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:55.138397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eaebd-e8f4-4571-95ec-d290af114554 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:55.138675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eaebd-e8f4-4571-95ec-d290af114554 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:55.138942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eaebd-e8f4-4571-95ec-d290af114554 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:55.140585 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b25eaebd-e8f4-4571-95ec-d290af114554 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:55.151082 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25eaebd-e8f4-4571-95ec-d290af114554 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:55.152486 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25eaebd-e8f4-4571-95ec-d290af114554 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:55.162319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1181/2353] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:57.400770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f03e2-f49f-46fe-b298-6798d4d66959 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:57.401757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f03e2-f49f-46fe-b298-6798d4d66959 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:57.402257 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f03e2-f49f-46fe-b298-6798d4d66959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:57.424816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-042f03e2-f49f-46fe-b298-6798d4d66959 None None] MFA Rules not processed for user `ab1a288170374d1a80355c869c78b4ca`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:57.426033 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-042f03e2-f49f-46fe-b298-6798d4d66959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:57.433295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1173/2354] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:56:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:57:05.157699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=60 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:05.788669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25eaebd-e8f4-4571-95ec-d290af114554 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:05.790063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b25eaebd-e8f4-4571-95ec-d290af114554 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:05.806079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25eaebd-e8f4-4571-95ec-d290af114554 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:05.812676 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c1638-fa70-44a1-9eed-ee441e17f814 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:05.812676 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c1638-fa70-44a1-9eed-ee441e17f814 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:05.812676 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c1638-fa70-44a1-9eed-ee441e17f814 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:05.812992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c1638-fa70-44a1-9eed-ee441e17f814 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:05.814522 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-965c1638-fa70-44a1-9eed-ee441e17f814 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:05.824912 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c1638-fa70-44a1-9eed-ee441e17f814 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:05.826536 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c1638-fa70-44a1-9eed-ee441e17f814 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:05.835814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1182/2355] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:06.771031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f03e2-f49f-46fe-b298-6798d4d66959 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:06.780821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f03e2-f49f-46fe-b298-6798d4d66959 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:06.784849 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75acd97-00ae-43f3-899b-ee7836ef7928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:06.785459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75acd97-00ae-43f3-899b-ee7836ef7928 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:06.785762 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75acd97-00ae-43f3-899b-ee7836ef7928 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:06.786026 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75acd97-00ae-43f3-899b-ee7836ef7928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:06.787872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c75acd97-00ae-43f3-899b-ee7836ef7928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:06.802784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75acd97-00ae-43f3-899b-ee7836ef7928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:06.804221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75acd97-00ae-43f3-899b-ee7836ef7928 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:06.813291 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1174/2356] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:15.831621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=6 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:16.809209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=40 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:21.590501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c1638-fa70-44a1-9eed-ee441e17f814 service nova] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:21.591939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-965c1638-fa70-44a1-9eed-ee441e17f814 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:21.606546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c1638-fa70-44a1-9eed-ee441e17f814 service nova] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:21.612630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9ac34-e4ef-423b-ab8e-01e022abe795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:21.613249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9ac34-e4ef-423b-ab8e-01e022abe795 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:21.613531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9ac34-e4ef-423b-ab8e-01e022abe795 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:21.613798 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9ac34-e4ef-423b-ab8e-01e022abe795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:21.616174 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-29d9ac34-e4ef-423b-ab8e-01e022abe795 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:21.629963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9ac34-e4ef-423b-ab8e-01e022abe795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:21.631756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9ac34-e4ef-423b-ab8e-01e022abe795 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:21.644233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1183/2357] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:23.363528 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75acd97-00ae-43f3-899b-ee7836ef7928 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:23.372321 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75acd97-00ae-43f3-899b-ee7836ef7928 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:23.380971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61959f4-f379-421c-8676-354b73be9049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:23.381615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61959f4-f379-421c-8676-354b73be9049 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:23.382088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61959f4-f379-421c-8676-354b73be9049 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:23.382520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61959f4-f379-421c-8676-354b73be9049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:23.384450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b61959f4-f379-421c-8676-354b73be9049 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:23.397292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61959f4-f379-421c-8676-354b73be9049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:23.398716 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61959f4-f379-421c-8676-354b73be9049 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:23.410144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1175/2358] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:26.464243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bffb2-614f-4340-a5cf-0072b7d2202b None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:26.464778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bffb2-614f-4340-a5cf-0072b7d2202b None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:26.464949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bffb2-614f-4340-a5cf-0072b7d2202b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:26.485864 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9f0bffb2-614f-4340-a5cf-0072b7d2202b None None] MFA Rules not processed for user `aee01beeba5b48c3b6c25489743de40e`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:26.487097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9f0bffb2-614f-4340-a5cf-0072b7d2202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:26.493999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1184/2359] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:57:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:26.934471 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61959f4-f379-421c-8676-354b73be9049 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:26.944559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61959f4-f379-421c-8676-354b73be9049 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:26.948939 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b8563-fcbd-4a7e-bdbb-de3585387686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:26.949633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b8563-fcbd-4a7e-bdbb-de3585387686 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:26.949983 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b8563-fcbd-4a7e-bdbb-de3585387686 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:26.950174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b8563-fcbd-4a7e-bdbb-de3585387686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:26.952279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fb1b8563-fcbd-4a7e-bdbb-de3585387686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:26.963953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b8563-fcbd-4a7e-bdbb-de3585387686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:26.965542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b8563-fcbd-4a7e-bdbb-de3585387686 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:26.974397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1176/2360] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:27.113508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bffb2-614f-4340-a5cf-0072b7d2202b None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:27.125784 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bffb2-614f-4340-a5cf-0072b7d2202b None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:27.129392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c6637-0d02-478a-9bb2-9f07535ce1a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:27.129995 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c6637-0d02-478a-9bb2-9f07535ce1a1 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:27.130278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c6637-0d02-478a-9bb2-9f07535ce1a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:27.130546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c6637-0d02-478a-9bb2-9f07535ce1a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:27.132247 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c74c6637-0d02-478a-9bb2-9f07535ce1a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:27.143654 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c6637-0d02-478a-9bb2-9f07535ce1a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:27.145254 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c6637-0d02-478a-9bb2-9f07535ce1a1 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:27.153514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1185/2361] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:28.291815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b8563-fcbd-4a7e-bdbb-de3585387686 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:28.293551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fb1b8563-fcbd-4a7e-bdbb-de3585387686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:28.425548 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b8563-fcbd-4a7e-bdbb-de3585387686 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:28.430719 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b21d9-3254-44a0-a67b-bfff3c1e8931 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:28.431326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b21d9-3254-44a0-a67b-bfff3c1e8931 service placement] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:28.431612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b21d9-3254-44a0-a67b-bfff3c1e8931 service placement] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:28.431891 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b21d9-3254-44a0-a67b-bfff3c1e8931 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:28.433663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4b7b21d9-3254-44a0-a67b-bfff3c1e8931 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:28.445380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b21d9-3254-44a0-a67b-bfff3c1e8931 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:28.446971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b21d9-3254-44a0-a67b-bfff3c1e8931 service placement] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:28.457606 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1177/2362] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:57:28.513959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c6637-0d02-478a-9bb2-9f07535ce1a1 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:28.515414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c74c6637-0d02-478a-9bb2-9f07535ce1a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:28.543721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c6637-0d02-478a-9bb2-9f07535ce1a1 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:28.549000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f28a1-3cb7-4c73-8915-285ee34d5cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:28.549641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f28a1-3cb7-4c73-8915-285ee34d5cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:28.549926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f28a1-3cb7-4c73-8915-285ee34d5cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:28.550201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f28a1-3cb7-4c73-8915-285ee34d5cf9 admin admin] PATH_INFO: `/v3/users/e23edadcfe3d4a0b81b13e278682ca2f` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:28.551513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f28a1-3cb7-4c73-8915-285ee34d5cf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e23edadcfe3d4a0b81b13e278682ca2f)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:28.553415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f28a1-3cb7-4c73-8915-285ee34d5cf9 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:28.814018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-755f28a1-3cb7-4c73-8915-285ee34d5cf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e23edadcfe3d4a0b81b13e278682ca2f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:28.821487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-755f28a1-3cb7-4c73-8915-285ee34d5cf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e23edadcfe3d4a0b81b13e278682ca2f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:28.971759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-755f28a1-3cb7-4c73-8915-285ee34d5cf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e23edadcfe3d4a0b81b13e278682ca2f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:28.972869 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-755f28a1-3cb7-4c73-8915-285ee34d5cf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e23edadcfe3d4a0b81b13e278682ca2f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:28.998813 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-755f28a1-3cb7-4c73-8915-285ee34d5cf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e23edadcfe3d4a0b81b13e278682ca2f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:29.008017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1186/2363] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 07:57:28 2022] DELETE /identity/v3/users/e23edadcfe3d4a0b81b13e278682ca2f => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:29.603082 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b21d9-3254-44a0-a67b-bfff3c1e8931 service placement] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:29.620935 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b21d9-3254-44a0-a67b-bfff3c1e8931 service placement] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:29.698218 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb70ec-9138-4275-a702-90f9034e32f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:29.699123 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb70ec-9138-4275-a702-90f9034e32f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:29.699408 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb70ec-9138-4275-a702-90f9034e32f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:29.699713 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb70ec-9138-4275-a702-90f9034e32f4 admin admin] PATH_INFO: `/v3/projects/a75ba813beae4999b9ae2afd58de8d25` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:29.700901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb70ec-9138-4275-a702-90f9034e32f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a75ba813beae4999b9ae2afd58de8d25)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:29.702947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb70ec-9138-4275-a702-90f9034e32f4 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:29.819915 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-60cb70ec-9138-4275-a702-90f9034e32f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a75ba813beae4999b9ae2afd58de8d25', 'request_id': 'req-60cb70ec-9138-4275-a702-90f9034e32f4'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:29.827070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-60cb70ec-9138-4275-a702-90f9034e32f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a75ba813beae4999b9ae2afd58de8d25', 'request_id': 'req-60cb70ec-9138-4275-a702-90f9034e32f4'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:29.932368 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-60cb70ec-9138-4275-a702-90f9034e32f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a75ba813beae4999b9ae2afd58de8d25', 'request_id': 'req-60cb70ec-9138-4275-a702-90f9034e32f4'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:29.936766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:29.944723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-60cb70ec-9138-4275-a702-90f9034e32f4 admin admin] The token cache is being invalidate because project a75ba813beae4999b9ae2afd58de8d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:57:29.945012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-60cb70ec-9138-4275-a702-90f9034e32f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:29.945646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-60cb70ec-9138-4275-a702-90f9034e32f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:29.947201 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1178/2364] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 07:57:29 2022] DELETE /identity/v3/projects/a75ba813beae4999b9ae2afd58de8d25 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:29.952497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e74fb-81fe-4b97-b5a3-e3fcf97223b7 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:29.954083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e74fb-81fe-4b97-b5a3-e3fcf97223b7 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:29.954083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e74fb-81fe-4b97-b5a3-e3fcf97223b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:29.974871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-d91e74fb-81fe-4b97-b5a3-e3fcf97223b7 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:29.976143 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d91e74fb-81fe-4b97-b5a3-e3fcf97223b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:30.054089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1187/2365] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:30.061612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9941543-4c9b-400e-b370-003cd0b66970 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:30.062006 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9941543-4c9b-400e-b370-003cd0b66970 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:30.062311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9941543-4c9b-400e-b370-003cd0b66970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:30.084626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d9941543-4c9b-400e-b370-003cd0b66970 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:30.085866 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d9941543-4c9b-400e-b370-003cd0b66970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:30.094624 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1179/2366] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:30.098970 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e74fb-81fe-4b97-b5a3-e3fcf97223b7 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:30.108705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e74fb-81fe-4b97-b5a3-e3fcf97223b7 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:30.114478 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f1a3-9f85-45b3-9740-f807bfe50be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:30.115085 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f1a3-9f85-45b3-9740-f807bfe50be3 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:30.115392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f1a3-9f85-45b3-9740-f807bfe50be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:30.115648 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f1a3-9f85-45b3-9740-f807bfe50be3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:30.116294 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137f1a3-9f85-45b3-9740-f807bfe50be3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:57:30.116631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137f1a3-9f85-45b3-9740-f807bfe50be3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:30.118333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137f1a3-9f85-45b3-9740-f807bfe50be3 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:30.130638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1188/2367] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 07:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:57:30.135210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9941543-4c9b-400e-b370-003cd0b66970 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:30.145346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9941543-4c9b-400e-b370-003cd0b66970 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:30.150106 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d99ac-83f5-42dc-84d9-cc6066f1b719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:30.150697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d99ac-83f5-42dc-84d9-cc6066f1b719 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:30.150984 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d99ac-83f5-42dc-84d9-cc6066f1b719 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:30.151257 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d99ac-83f5-42dc-84d9-cc6066f1b719 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:30.151888 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d99ac-83f5-42dc-84d9-cc6066f1b719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:30.153845 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d99ac-83f5-42dc-84d9-cc6066f1b719 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:30.240365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1180/2368] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 07:57:30 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:57:30.244953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f1a3-9f85-45b3-9740-f807bfe50be3 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:30.255387 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f1a3-9f85-45b3-9740-f807bfe50be3 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:30.279756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:30.311405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa74b4a-30e4-43ce-9e4b-45eb89a881b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:30.312086 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa74b4a-30e4-43ce-9e4b-45eb89a881b5 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:30.313008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa74b4a-30e4-43ce-9e4b-45eb89a881b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:30.313281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa74b4a-30e4-43ce-9e4b-45eb89a881b5 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:30.314001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa74b4a-30e4-43ce-9e4b-45eb89a881b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:30.315916 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa74b4a-30e4-43ce-9e4b-45eb89a881b5 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:30.417496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1189/2369] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 07:57:30 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:57:30.421758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d99ac-83f5-42dc-84d9-cc6066f1b719 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:30.432088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d99ac-83f5-42dc-84d9-cc6066f1b719 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:30.437041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ffc13-5e2a-4615-afb0-6fd9d0f4376a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:30.438275 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ffc13-5e2a-4615-afb0-6fd9d0f4376a admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:30.438737 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ffc13-5e2a-4615-afb0-6fd9d0f4376a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:30.439066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ffc13-5e2a-4615-afb0-6fd9d0f4376a admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:30.439704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ffc13-5e2a-4615-afb0-6fd9d0f4376a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:30.441339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ffc13-5e2a-4615-afb0-6fd9d0f4376a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:30.452258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1181/2370] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:30.456612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa74b4a-30e4-43ce-9e4b-45eb89a881b5 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:30.466367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa74b4a-30e4-43ce-9e4b-45eb89a881b5 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:30.471002 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bed963-a09f-47ee-90ee-2b6c50f90f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:30.471649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bed963-a09f-47ee-90ee-2b6c50f90f48 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:30.471869 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bed963-a09f-47ee-90ee-2b6c50f90f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:30.472148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bed963-a09f-47ee-90ee-2b6c50f90f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:30.472889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bed963-a09f-47ee-90ee-2b6c50f90f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:30.474472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bed963-a09f-47ee-90ee-2b6c50f90f48 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:30.488277 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1190/2371] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:30.489354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ffc13-5e2a-4615-afb0-6fd9d0f4376a admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:30.499433 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ffc13-5e2a-4615-afb0-6fd9d0f4376a admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:30.505245 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75956adc-96e0-48da-9880-53c41ee4443a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:30.505694 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75956adc-96e0-48da-9880-53c41ee4443a admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:30.506128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75956adc-96e0-48da-9880-53c41ee4443a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:30.506362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75956adc-96e0-48da-9880-53c41ee4443a admin admin] PATH_INFO: `/v3/projects/7f4908cc2cf74047bcd5cc1771b28189/users/1858e633724d446dbbd128b317158d3f/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:30.521685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75956adc-96e0-48da-9880-53c41ee4443a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f4908cc2cf74047bcd5cc1771b28189, user_id=1858e633724d446dbbd128b317158d3f, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:30.524490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75956adc-96e0-48da-9880-53c41ee4443a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:30.604897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1182/2372] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 07:57:30 2022] PUT /identity/v3/projects/7f4908cc2cf74047bcd5cc1771b28189/users/1858e633724d446dbbd128b317158d3f/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:30.612766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b5d3-889d-4e4a-9ad0-278baccd8280 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:30.612766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b5d3-889d-4e4a-9ad0-278baccd8280 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:30.612766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b5d3-889d-4e4a-9ad0-278baccd8280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:30.633494 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-85d7b5d3-889d-4e4a-9ad0-278baccd8280 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:30.634785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-85d7b5d3-889d-4e4a-9ad0-278baccd8280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:30.692874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1191/2373] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:30.702647 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75956adc-96e0-48da-9880-53c41ee4443a admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:30.704632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-75956adc-96e0-48da-9880-53c41ee4443a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:30.760389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75956adc-96e0-48da-9880-53c41ee4443a admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:30.764024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80883106-18e0-453b-b118-bd2567800695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:30.764617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80883106-18e0-453b-b118-bd2567800695 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:30.764925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80883106-18e0-453b-b118-bd2567800695 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:30.765199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80883106-18e0-453b-b118-bd2567800695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:30.773996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80883106-18e0-453b-b118-bd2567800695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:30.775419 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80883106-18e0-453b-b118-bd2567800695 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:30.784908 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1183/2374] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:36.601808 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712ad38-275b-4078-b3c7-89f0bd9a4623 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:36.602335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712ad38-275b-4078-b3c7-89f0bd9a4623 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:36.602335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712ad38-275b-4078-b3c7-89f0bd9a4623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:36.636159 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-d712ad38-275b-4078-b3c7-89f0bd9a4623 None None] MFA Rules not processed for user `1858e633724d446dbbd128b317158d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:36.637613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d712ad38-275b-4078-b3c7-89f0bd9a4623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:36.714507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1192/2375] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:57:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:36.760884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80883106-18e0-453b-b118-bd2567800695 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:36.760884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-80883106-18e0-453b-b118-bd2567800695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:36.839767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80883106-18e0-453b-b118-bd2567800695 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:36.846449 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe969c1-fe2d-49a2-8788-983ea4ac1337 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:36.847061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe969c1-fe2d-49a2-8788-983ea4ac1337 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:36.847344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe969c1-fe2d-49a2-8788-983ea4ac1337 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:36.847643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe969c1-fe2d-49a2-8788-983ea4ac1337 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:36.856711 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe969c1-fe2d-49a2-8788-983ea4ac1337 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:36.858870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe969c1-fe2d-49a2-8788-983ea4ac1337 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:36.869575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1184/2376] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:39.174410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c47a71-b986-4146-a67f-84d1f93fc8af None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:39.175139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c47a71-b986-4146-a67f-84d1f93fc8af None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:39.175418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c47a71-b986-4146-a67f-84d1f93fc8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:39.197491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-73c47a71-b986-4146-a67f-84d1f93fc8af None None] MFA Rules not processed for user `1858e633724d446dbbd128b317158d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:39.198887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-73c47a71-b986-4146-a67f-84d1f93fc8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:39.206499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1193/2377] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 07:57:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:46.864666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:DELETE=1,keystone:SELECT=89,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:49.191409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:20.998460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe969c1-fe2d-49a2-8788-983ea4ac1337 service nova] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:21.011673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe969c1-fe2d-49a2-8788-983ea4ac1337 service nova] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:21.017568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2409d-52b7-4e60-a9a0-37209c159eb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:21.018215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2409d-52b7-4e60-a9a0-37209c159eb5 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:21.018490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2409d-52b7-4e60-a9a0-37209c159eb5 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:21.018755 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2409d-52b7-4e60-a9a0-37209c159eb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:21.020542 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b7c2409d-52b7-4e60-a9a0-37209c159eb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:21.035534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2409d-52b7-4e60-a9a0-37209c159eb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:21.035534 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2409d-52b7-4e60-a9a0-37209c159eb5 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:21.046793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1185/2378] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:26.803283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c47a71-b986-4146-a67f-84d1f93fc8af None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:26.805112 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-73c47a71-b986-4146-a67f-84d1f93fc8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:26.822636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c47a71-b986-4146-a67f-84d1f93fc8af None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:26.828450 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69f097-5d8e-4054-8c65-85ea3420947b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:26.829099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69f097-5d8e-4054-8c65-85ea3420947b service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:26.832396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69f097-5d8e-4054-8c65-85ea3420947b service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:26.832396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69f097-5d8e-4054-8c65-85ea3420947b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:26.832396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ad69f097-5d8e-4054-8c65-85ea3420947b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:26.843928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69f097-5d8e-4054-8c65-85ea3420947b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:26.845539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69f097-5d8e-4054-8c65-85ea3420947b service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:26.855984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1194/2379] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:31.041678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=6 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:36.851552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=6 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:57.561102 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2409d-52b7-4e60-a9a0-37209c159eb5 service nova] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:57.572885 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2409d-52b7-4e60-a9a0-37209c159eb5 service nova] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:57.579495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f822c35-ac1a-4005-9363-b2cc210a4a41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:57.580101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f822c35-ac1a-4005-9363-b2cc210a4a41 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:57.580385 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f822c35-ac1a-4005-9363-b2cc210a4a41 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:57.580685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f822c35-ac1a-4005-9363-b2cc210a4a41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:57.582451 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2f822c35-ac1a-4005-9363-b2cc210a4a41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:57.643728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f822c35-ac1a-4005-9363-b2cc210a4a41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:57.645362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f822c35-ac1a-4005-9363-b2cc210a4a41 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:57.655651 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1186/2380] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:59:07.650338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=28 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:09.022728 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69f097-5d8e-4054-8c65-85ea3420947b service nova] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:09.034262 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69f097-5d8e-4054-8c65-85ea3420947b service nova] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:09.039912 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6bc36-a32f-40e7-b1f3-110856867c5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:09.040547 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6bc36-a32f-40e7-b1f3-110856867c5a service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:09.040895 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6bc36-a32f-40e7-b1f3-110856867c5a service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:09.041176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6bc36-a32f-40e7-b1f3-110856867c5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:09.042938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-26c6bc36-a32f-40e7-b1f3-110856867c5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:09.101366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6bc36-a32f-40e7-b1f3-110856867c5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:09.102945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6bc36-a32f-40e7-b1f3-110856867c5a service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:09.111849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1195/2381] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 07:59:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:59:19.107663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=28 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:00:08.337912 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f822c35-ac1a-4005-9363-b2cc210a4a41 service nova] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:00:08.339467 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2f822c35-ac1a-4005-9363-b2cc210a4a41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:00:08.494613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f822c35-ac1a-4005-9363-b2cc210a4a41 service nova] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:00:08.499850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e251d2-5386-488a-9cda-dee14373f00a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:00:08.500451 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e251d2-5386-488a-9cda-dee14373f00a service placement] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:00:08.500766 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e251d2-5386-488a-9cda-dee14373f00a service placement] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:00:08.501044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e251d2-5386-488a-9cda-dee14373f00a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:00:08.503085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-12e251d2-5386-488a-9cda-dee14373f00a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:00:08.514811 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e251d2-5386-488a-9cda-dee14373f00a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:00:08.516402 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e251d2-5386-488a-9cda-dee14373f00a service placement] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:00:08.530879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1187/2382] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:00:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:00:15.543780 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f4db8-d45e-4acf-9729-eecd51ed6c29 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:00:15.544301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f4db8-d45e-4acf-9729-eecd51ed6c29 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:00:15.544301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f4db8-d45e-4acf-9729-eecd51ed6c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:00:15.563542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-241f4db8-d45e-4acf-9729-eecd51ed6c29 None None] MFA Rules not processed for user `1858e633724d446dbbd128b317158d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:00:15.564979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-241f4db8-d45e-4acf-9729-eecd51ed6c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:00:15.572702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1196/2383] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:00:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:00:15.950413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e251d2-5386-488a-9cda-dee14373f00a service placement] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:00:15.969741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e251d2-5386-488a-9cda-dee14373f00a service placement] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:00:15.974086 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df230b-50f5-4ee1-a428-2b925ba557df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:00:15.974678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df230b-50f5-4ee1-a428-2b925ba557df service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:00:15.974962 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df230b-50f5-4ee1-a428-2b925ba557df service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:00:15.975241 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df230b-50f5-4ee1-a428-2b925ba557df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:00:15.977025 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d8df230b-50f5-4ee1-a428-2b925ba557df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:00:15.999258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df230b-50f5-4ee1-a428-2b925ba557df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:00:16.000710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df230b-50f5-4ee1-a428-2b925ba557df service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:00:16.008864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1188/2384] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:00:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:00:25.557816 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:00:26.005402 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=78 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:50.323143 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f4db8-d45e-4acf-9729-eecd51ed6c29 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:50.324634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-241f4db8-d45e-4acf-9729-eecd51ed6c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:50.336292 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f4db8-d45e-4acf-9729-eecd51ed6c29 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:50.340127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c1f40-880f-47d3-aa92-ed33422d901b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:50.340751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1f40-880f-47d3-aa92-ed33422d901b service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:50.341031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1f40-880f-47d3-aa92-ed33422d901b service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:50.341302 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1f40-880f-47d3-aa92-ed33422d901b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:50.342954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-683c1f40-880f-47d3-aa92-ed33422d901b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:50.375420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c1f40-880f-47d3-aa92-ed33422d901b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:50.376891 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c1f40-880f-47d3-aa92-ed33422d901b service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:50.397356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1197/2385] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:50.918746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df230b-50f5-4ee1-a428-2b925ba557df service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:50.927294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df230b-50f5-4ee1-a428-2b925ba557df service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:50.931223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b478e-295a-426c-8613-34c49ac507c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:50.931819 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b478e-295a-426c-8613-34c49ac507c3 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:50.932099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b478e-295a-426c-8613-34c49ac507c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:50.932373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b478e-295a-426c-8613-34c49ac507c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:50.934176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-773b478e-295a-426c-8613-34c49ac507c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:50.948027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b478e-295a-426c-8613-34c49ac507c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:50.949539 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b478e-295a-426c-8613-34c49ac507c3 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:50.960424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1189/2386] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:00.394784 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=18 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:00.955600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=6 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:17.691761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c1f40-880f-47d3-aa92-ed33422d901b service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:17.704159 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c1f40-880f-47d3-aa92-ed33422d901b service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:17.709452 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc992ad1-0722-460a-857d-27c39b28150e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:17.710130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc992ad1-0722-460a-857d-27c39b28150e service placement] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:17.710592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc992ad1-0722-460a-857d-27c39b28150e service placement] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:17.710939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc992ad1-0722-460a-857d-27c39b28150e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:17.713875 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-bc992ad1-0722-460a-857d-27c39b28150e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:17.725709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc992ad1-0722-460a-857d-27c39b28150e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:17.728182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc992ad1-0722-460a-857d-27c39b28150e service placement] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:17.737907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1198/2387] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:17.828495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b478e-295a-426c-8613-34c49ac507c3 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:17.860306 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b478e-295a-426c-8613-34c49ac507c3 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:17.864917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d1a29-dc3c-451b-835a-7e63295fed86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:17.865573 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d1a29-dc3c-451b-835a-7e63295fed86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:17.865856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d1a29-dc3c-451b-835a-7e63295fed86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:17.866134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d1a29-dc3c-451b-835a-7e63295fed86 admin admin] PATH_INFO: `/v3/users/1858e633724d446dbbd128b317158d3f` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:17.867286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d1a29-dc3c-451b-835a-7e63295fed86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1858e633724d446dbbd128b317158d3f)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:17.869318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d1a29-dc3c-451b-835a-7e63295fed86 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:18.108369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-957d1a29-dc3c-451b-835a-7e63295fed86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1858e633724d446dbbd128b317158d3f'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:18.198099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-957d1a29-dc3c-451b-835a-7e63295fed86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1858e633724d446dbbd128b317158d3f'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:18.198733 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-957d1a29-dc3c-451b-835a-7e63295fed86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1858e633724d446dbbd128b317158d3f'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:18.206412 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-957d1a29-dc3c-451b-835a-7e63295fed86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1858e633724d446dbbd128b317158d3f'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:18.219130 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-957d1a29-dc3c-451b-835a-7e63295fed86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1858e633724d446dbbd128b317158d3f'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:18.225230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1190/2388] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:02:17 2022] DELETE /identity/v3/users/1858e633724d446dbbd128b317158d3f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:18.552810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc992ad1-0722-460a-857d-27c39b28150e service placement] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:18.556916 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d1a29-dc3c-451b-835a-7e63295fed86 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:18.558435 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-957d1a29-dc3c-451b-835a-7e63295fed86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:18.568470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc992ad1-0722-460a-857d-27c39b28150e service placement] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:18.629375 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6fb19-000c-46f0-9853-9c942a0f1c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:18.630012 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6fb19-000c-46f0-9853-9c942a0f1c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:18.630296 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6fb19-000c-46f0-9853-9c942a0f1c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:18.630571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6fb19-000c-46f0-9853-9c942a0f1c1f admin admin] PATH_INFO: `/v3/projects/7f4908cc2cf74047bcd5cc1771b28189` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:18.631685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6fb19-000c-46f0-9853-9c942a0f1c1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f4908cc2cf74047bcd5cc1771b28189)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:18.633726 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6fb19-000c-46f0-9853-9c942a0f1c1f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:18.656531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d1a29-dc3c-451b-835a-7e63295fed86 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:18.660323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52984f8b-c461-4f85-807d-0ae08ca35b08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:18.660975 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52984f8b-c461-4f85-807d-0ae08ca35b08 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:18.661262 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52984f8b-c461-4f85-807d-0ae08ca35b08 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:18.661543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52984f8b-c461-4f85-807d-0ae08ca35b08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:18.663305 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-52984f8b-c461-4f85-807d-0ae08ca35b08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:18.718396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:18.729494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52984f8b-c461-4f85-807d-0ae08ca35b08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:18.730915 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52984f8b-c461-4f85-807d-0ae08ca35b08 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:18.740690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1191/2389] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:18.741479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52984f8b-c461-4f85-807d-0ae08ca35b08 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:18.743459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-52984f8b-c461-4f85-807d-0ae08ca35b08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:18.753764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52984f8b-c461-4f85-807d-0ae08ca35b08 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:18.757511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefcb6f-ac2b-4302-8469-32174fbfd293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:18.758224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefcb6f-ac2b-4302-8469-32174fbfd293 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:18.758597 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefcb6f-ac2b-4302-8469-32174fbfd293 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:18.759094 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefcb6f-ac2b-4302-8469-32174fbfd293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:18.760682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9aefcb6f-ac2b-4302-8469-32174fbfd293 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:18.781166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-23a6fb19-000c-46f0-9853-9c942a0f1c1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f4908cc2cf74047bcd5cc1771b28189', 'request_id': 'req-23a6fb19-000c-46f0-9853-9c942a0f1c1f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:18.787283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-23a6fb19-000c-46f0-9853-9c942a0f1c1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f4908cc2cf74047bcd5cc1771b28189', 'request_id': 'req-23a6fb19-000c-46f0-9853-9c942a0f1c1f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:18.794111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-23a6fb19-000c-46f0-9853-9c942a0f1c1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f4908cc2cf74047bcd5cc1771b28189', 'request_id': 'req-23a6fb19-000c-46f0-9853-9c942a0f1c1f'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:18.843174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefcb6f-ac2b-4302-8469-32174fbfd293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:18.844535 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefcb6f-ac2b-4302-8469-32174fbfd293 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:18.863531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1192/2390] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:18.864325 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefcb6f-ac2b-4302-8469-32174fbfd293 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:18.872366 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefcb6f-ac2b-4302-8469-32174fbfd293 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:18.933386 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-23a6fb19-000c-46f0-9853-9c942a0f1c1f admin admin] The token cache is being invalidate because project 7f4908cc2cf74047bcd5cc1771b28189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:02:18.933923 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-23a6fb19-000c-46f0-9853-9c942a0f1c1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:18.934796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-23a6fb19-000c-46f0-9853-9c942a0f1c1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:18.936667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1199/2391] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:02:18 2022] DELETE /identity/v3/projects/7f4908cc2cf74047bcd5cc1771b28189 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:18.936922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc25f6-f6f9-4a8f-8103-e6ad9ab0c165 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:18.937580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc25f6-f6f9-4a8f-8103-e6ad9ab0c165 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:18.937880 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc25f6-f6f9-4a8f-8103-e6ad9ab0c165 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:18.938137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc25f6-f6f9-4a8f-8103-e6ad9ab0c165 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:18.940123 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-dcfc25f6-f6f9-4a8f-8103-e6ad9ab0c165 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:18.941792 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ac741-6fbb-4590-81d3-d06f8cfedd02 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:18.942287 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ac741-6fbb-4590-81d3-d06f8cfedd02 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:18.943030 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ac741-6fbb-4590-81d3-d06f8cfedd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:19.015262 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-4a5ac741-6fbb-4590-81d3-d06f8cfedd02 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:19.016582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4a5ac741-6fbb-4590-81d3-d06f8cfedd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:19.028681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc25f6-f6f9-4a8f-8103-e6ad9ab0c165 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:19.031140 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc25f6-f6f9-4a8f-8103-e6ad9ab0c165 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:19.050298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:19.054934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1193/2392] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:19.081238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1200/2393] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:19.090261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f7ba3-f2f8-46a7-8251-5f969f9d01f8 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:19.095285 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f7ba3-f2f8-46a7-8251-5f969f9d01f8 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:19.095285 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f7ba3-f2f8-46a7-8251-5f969f9d01f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:19.110702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-986f7ba3-f2f8-46a7-8251-5f969f9d01f8 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:19.111910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-986f7ba3-f2f8-46a7-8251-5f969f9d01f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:19.120919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1194/2394] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:19.125185 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ac741-6fbb-4590-81d3-d06f8cfedd02 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:19.135811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ac741-6fbb-4590-81d3-d06f8cfedd02 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:19.140906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893c51b-2254-4ddc-9436-c4a63c67780b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:19.141574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893c51b-2254-4ddc-9436-c4a63c67780b admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:19.141867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893c51b-2254-4ddc-9436-c4a63c67780b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:19.142144 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893c51b-2254-4ddc-9436-c4a63c67780b admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:19.142796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893c51b-2254-4ddc-9436-c4a63c67780b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:02:19.143124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893c51b-2254-4ddc-9436-c4a63c67780b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:19.145002 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893c51b-2254-4ddc-9436-c4a63c67780b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:19.170563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1201/2395] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:02:19.170784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f7ba3-f2f8-46a7-8251-5f969f9d01f8 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:19.175286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f7ba3-f2f8-46a7-8251-5f969f9d01f8 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:19.180301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c267863-d5ef-41ec-b77e-19544967d5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:19.181517 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c267863-d5ef-41ec-b77e-19544967d5af admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:19.181989 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c267863-d5ef-41ec-b77e-19544967d5af admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:19.182636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c267863-d5ef-41ec-b77e-19544967d5af admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:19.183668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c267863-d5ef-41ec-b77e-19544967d5af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:19.185793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c267863-d5ef-41ec-b77e-19544967d5af admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:19.302887 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:19.310077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1195/2396] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:02:19 2022] POST /identity/v3/projects => generated 403 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:02:19.314261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893c51b-2254-4ddc-9436-c4a63c67780b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:19.325009 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893c51b-2254-4ddc-9436-c4a63c67780b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:19.380001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9a0bb-0469-4317-a4d4-b9f86739dae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:19.380527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9a0bb-0469-4317-a4d4-b9f86739dae3 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:19.380840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9a0bb-0469-4317-a4d4-b9f86739dae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:19.381119 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9a0bb-0469-4317-a4d4-b9f86739dae3 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:19.381736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9a0bb-0469-4317-a4d4-b9f86739dae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:19.383666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9a0bb-0469-4317-a4d4-b9f86739dae3 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:19.521481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1202/2397] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:02:19 2022] POST /identity/v3/users => generated 302 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:19.527006 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c267863-d5ef-41ec-b77e-19544967d5af admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:19.533509 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c267863-d5ef-41ec-b77e-19544967d5af admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:19.538567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150643bb-15ac-46f9-873d-b61c0ee57c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:19.539255 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150643bb-15ac-46f9-873d-b61c0ee57c6b admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:19.539450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150643bb-15ac-46f9-873d-b61c0ee57c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:19.539728 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150643bb-15ac-46f9-873d-b61c0ee57c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:19.540450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150643bb-15ac-46f9-873d-b61c0ee57c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:19.542128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150643bb-15ac-46f9-873d-b61c0ee57c6b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:19.552293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1196/2398] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:19.556903 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9a0bb-0469-4317-a4d4-b9f86739dae3 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:19.573067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9a0bb-0469-4317-a4d4-b9f86739dae3 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:19.573067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25eb175-329a-42f0-ad9f-3b638d3f49c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:19.573709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25eb175-329a-42f0-ad9f-3b638d3f49c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:19.573991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25eb175-329a-42f0-ad9f-3b638d3f49c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:19.574267 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25eb175-329a-42f0-ad9f-3b638d3f49c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:19.574914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25eb175-329a-42f0-ad9f-3b638d3f49c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:19.576508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25eb175-329a-42f0-ad9f-3b638d3f49c5 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:19.588049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1203/2399] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:19.596208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150643bb-15ac-46f9-873d-b61c0ee57c6b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:19.605066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150643bb-15ac-46f9-873d-b61c0ee57c6b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:19.610067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003e581-37f5-4cff-9eef-d5240f77ec73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:19.610722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003e581-37f5-4cff-9eef-d5240f77ec73 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:19.611436 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003e581-37f5-4cff-9eef-d5240f77ec73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:19.611436 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003e581-37f5-4cff-9eef-d5240f77ec73 admin admin] PATH_INFO: `/v3/projects/51a560f652ab4ff9b0748a26425b4d7a/users/6dc5b07383ac450daabc5ab58cbe5c1e/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:19.631351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003e581-37f5-4cff-9eef-d5240f77ec73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a560f652ab4ff9b0748a26425b4d7a, user_id=6dc5b07383ac450daabc5ab58cbe5c1e, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:19.633852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003e581-37f5-4cff-9eef-d5240f77ec73 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:19.768537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1197/2400] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:19 2022] PUT /identity/v3/projects/51a560f652ab4ff9b0748a26425b4d7a/users/6dc5b07383ac450daabc5ab58cbe5c1e/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:19.773847 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9614d-2496-45cc-9ffa-1f58d058bcd5 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:19.774229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9614d-2496-45cc-9ffa-1f58d058bcd5 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:19.774493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9614d-2496-45cc-9ffa-1f58d058bcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:19.793088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-53b9614d-2496-45cc-9ffa-1f58d058bcd5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:19.794290 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-53b9614d-2496-45cc-9ffa-1f58d058bcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:19.855595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1204/2401] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:19.865323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003e581-37f5-4cff-9eef-d5240f77ec73 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:19.867203 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e003e581-37f5-4cff-9eef-d5240f77ec73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:19.924486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003e581-37f5-4cff-9eef-d5240f77ec73 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:19.928522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94045d5-b0e8-431e-b20c-f401eef98e18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:19.929160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94045d5-b0e8-431e-b20c-f401eef98e18 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:19.929446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94045d5-b0e8-431e-b20c-f401eef98e18 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:19.929724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94045d5-b0e8-431e-b20c-f401eef98e18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:19.939983 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94045d5-b0e8-431e-b20c-f401eef98e18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:19.941466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94045d5-b0e8-431e-b20c-f401eef98e18 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:19.951333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1198/2402] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:21.335783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd4ee9-a22e-43e7-adf9-cf392870f394 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:21.336198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd4ee9-a22e-43e7-adf9-cf392870f394 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:21.336479 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd4ee9-a22e-43e7-adf9-cf392870f394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:21.365965 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-1cbd4ee9-a22e-43e7-adf9-cf392870f394 None None] MFA Rules not processed for user `6dc5b07383ac450daabc5ab58cbe5c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:21.367130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-1cbd4ee9-a22e-43e7-adf9-cf392870f394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:21.398750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:21.434288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1205/2403] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:02:21 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:21.439444 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94045d5-b0e8-431e-b20c-f401eef98e18 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:21.452632 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94045d5-b0e8-431e-b20c-f401eef98e18 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:21.457468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09645aef-f082-48a5-a6c7-1f9004902ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:21.458065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09645aef-f082-48a5-a6c7-1f9004902ccc admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:21.458352 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09645aef-f082-48a5-a6c7-1f9004902ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:21.458620 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09645aef-f082-48a5-a6c7-1f9004902ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:21.459393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09645aef-f082-48a5-a6c7-1f9004902ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:21.461198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09645aef-f082-48a5-a6c7-1f9004902ccc admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:21.552217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1199/2404] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:02:21 2022] POST /identity/v3/projects => generated 401 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:21.556930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd4ee9-a22e-43e7-adf9-cf392870f394 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:21.567734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd4ee9-a22e-43e7-adf9-cf392870f394 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:21.624942 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44e498-8e99-46dd-93f0-e87b06a39f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:21.625551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44e498-8e99-46dd-93f0-e87b06a39f0c admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:21.625835 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44e498-8e99-46dd-93f0-e87b06a39f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:21.626106 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44e498-8e99-46dd-93f0-e87b06a39f0c admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:21.626739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44e498-8e99-46dd-93f0-e87b06a39f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:21.628669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44e498-8e99-46dd-93f0-e87b06a39f0c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:21.704569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1206/2405] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:02:21 2022] POST /identity/v3/users => generated 307 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:21.708628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09645aef-f082-48a5-a6c7-1f9004902ccc admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:21.719353 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09645aef-f082-48a5-a6c7-1f9004902ccc admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:21.723910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310ae96-056f-4d4c-b180-a2c4c57b633e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:21.724500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310ae96-056f-4d4c-b180-a2c4c57b633e admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:21.724817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310ae96-056f-4d4c-b180-a2c4c57b633e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:21.725088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310ae96-056f-4d4c-b180-a2c4c57b633e admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:21.725740 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310ae96-056f-4d4c-b180-a2c4c57b633e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:21.727349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310ae96-056f-4d4c-b180-a2c4c57b633e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:21.739505 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1200/2406] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:21.745023 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44e498-8e99-46dd-93f0-e87b06a39f0c admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:21.754989 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44e498-8e99-46dd-93f0-e87b06a39f0c admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:21.759823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3778be5-b13e-4a0d-af77-dd8ede15e6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:21.760465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3778be5-b13e-4a0d-af77-dd8ede15e6cc admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:21.760776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3778be5-b13e-4a0d-af77-dd8ede15e6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:21.761055 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3778be5-b13e-4a0d-af77-dd8ede15e6cc admin admin] PATH_INFO: `/v3/projects/1225d5b2b1574d59afb9a614fb0488bc/users/b918776128de40ec94d252b59e278651/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:21.776581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3778be5-b13e-4a0d-af77-dd8ede15e6cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1225d5b2b1574d59afb9a614fb0488bc, user_id=b918776128de40ec94d252b59e278651, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:21.779318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3778be5-b13e-4a0d-af77-dd8ede15e6cc admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:21.883656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1207/2407] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:21 2022] PUT /identity/v3/projects/1225d5b2b1574d59afb9a614fb0488bc/users/b918776128de40ec94d252b59e278651/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:21.887746 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310ae96-056f-4d4c-b180-a2c4c57b633e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:21.897599 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310ae96-056f-4d4c-b180-a2c4c57b633e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:21.941398 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:21.956229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523271e0-e9b8-4130-a805-dd2810bd2467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:21.957928 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523271e0-e9b8-4130-a805-dd2810bd2467 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:21.957928 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523271e0-e9b8-4130-a805-dd2810bd2467 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:21.957928 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523271e0-e9b8-4130-a805-dd2810bd2467 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:21.958210 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523271e0-e9b8-4130-a805-dd2810bd2467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:21.959670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523271e0-e9b8-4130-a805-dd2810bd2467 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:21.973667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1201/2408] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:21.976731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3778be5-b13e-4a0d-af77-dd8ede15e6cc admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:21.987891 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3778be5-b13e-4a0d-af77-dd8ede15e6cc admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:21.992816 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966901e2-9f5a-40a8-bfdb-375d401f0184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:21.993477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966901e2-9f5a-40a8-bfdb-375d401f0184 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:21.998615 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966901e2-9f5a-40a8-bfdb-375d401f0184 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:21.998615 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966901e2-9f5a-40a8-bfdb-375d401f0184 admin admin] PATH_INFO: `/v3/projects/1225d5b2b1574d59afb9a614fb0488bc/users/b918776128de40ec94d252b59e278651/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:21.998615 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966901e2-9f5a-40a8-bfdb-375d401f0184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1225d5b2b1574d59afb9a614fb0488bc, user_id=b918776128de40ec94d252b59e278651, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:21.998920 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966901e2-9f5a-40a8-bfdb-375d401f0184 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:22.010491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1208/2409] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:21 2022] PUT /identity/v3/projects/1225d5b2b1574d59afb9a614fb0488bc/users/b918776128de40ec94d252b59e278651/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:23.383920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c79eaf-74da-41f6-bcc1-c21ec7514340 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:23.385238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c79eaf-74da-41f6-bcc1-c21ec7514340 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:23.385238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c79eaf-74da-41f6-bcc1-c21ec7514340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:23.414109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-49c79eaf-74da-41f6-bcc1-c21ec7514340 None None] MFA Rules not processed for user `b918776128de40ec94d252b59e278651`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:23.415344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-49c79eaf-74da-41f6-bcc1-c21ec7514340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:23.487628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1202/2410] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:02:23 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:24.014888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966901e2-9f5a-40a8-bfdb-375d401f0184 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:24.016785 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-966901e2-9f5a-40a8-bfdb-375d401f0184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:24.089036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966901e2-9f5a-40a8-bfdb-375d401f0184 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:24.094108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7fa2f-9de5-4e98-bb29-bcbc9cffa978 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:24.094710 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7fa2f-9de5-4e98-bb29-bcbc9cffa978 service placement] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:24.094995 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7fa2f-9de5-4e98-bb29-bcbc9cffa978 service placement] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:24.095273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7fa2f-9de5-4e98-bb29-bcbc9cffa978 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:24.097054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-cdd7fa2f-9de5-4e98-bb29-bcbc9cffa978 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:24.108359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7fa2f-9de5-4e98-bb29-bcbc9cffa978 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:24.109988 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7fa2f-9de5-4e98-bb29-bcbc9cffa978 service placement] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:24.120229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1209/2411] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:24.804085 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c79eaf-74da-41f6-bcc1-c21ec7514340 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:24.867646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c79eaf-74da-41f6-bcc1-c21ec7514340 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:24.872277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb7164-27a0-4c97-8b23-58b044f1e9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:24.872928 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb7164-27a0-4c97-8b23-58b044f1e9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:24.873213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb7164-27a0-4c97-8b23-58b044f1e9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:24.873486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb7164-27a0-4c97-8b23-58b044f1e9fe admin admin] PATH_INFO: `/v3/users/6dc5b07383ac450daabc5ab58cbe5c1e` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:24.874662 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb7164-27a0-4c97-8b23-58b044f1e9fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dc5b07383ac450daabc5ab58cbe5c1e)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:24.876576 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb7164-27a0-4c97-8b23-58b044f1e9fe admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:25.108393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f1bb7164-27a0-4c97-8b23-58b044f1e9fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dc5b07383ac450daabc5ab58cbe5c1e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:25.191825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f1bb7164-27a0-4c97-8b23-58b044f1e9fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dc5b07383ac450daabc5ab58cbe5c1e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:25.192729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f1bb7164-27a0-4c97-8b23-58b044f1e9fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dc5b07383ac450daabc5ab58cbe5c1e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:25.200712 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f1bb7164-27a0-4c97-8b23-58b044f1e9fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dc5b07383ac450daabc5ab58cbe5c1e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:25.205105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:25.214053 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f1bb7164-27a0-4c97-8b23-58b044f1e9fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dc5b07383ac450daabc5ab58cbe5c1e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:25.219920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1203/2412] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:02:24 2022] DELETE /identity/v3/users/6dc5b07383ac450daabc5ab58cbe5c1e => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:25.794064 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7fa2f-9de5-4e98-bb29-bcbc9cffa978 service placement] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:25.805018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7fa2f-9de5-4e98-bb29-bcbc9cffa978 service placement] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:25.821176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:25.869393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:25.870316 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:25.870762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:25.871037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] PATH_INFO: `/v3/projects/51a560f652ab4ff9b0748a26425b4d7a` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:25.872213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] RBAC: Authorizing `identity:delete_project(project_id=51a560f652ab4ff9b0748a26425b4d7a)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:25.874160 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:26.026907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51a560f652ab4ff9b0748a26425b4d7a', 'request_id': 'req-e4b861e7-97fe-4f64-9a3a-da7377b5139e'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:26.033056 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51a560f652ab4ff9b0748a26425b4d7a', 'request_id': 'req-e4b861e7-97fe-4f64-9a3a-da7377b5139e'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:26.038471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51a560f652ab4ff9b0748a26425b4d7a', 'request_id': 'req-e4b861e7-97fe-4f64-9a3a-da7377b5139e'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:26.174235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] The token cache is being invalidate because project 51a560f652ab4ff9b0748a26425b4d7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:02:26.174528 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:26.175165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:26.176809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1210/2413] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:02:25 2022] DELETE /identity/v3/projects/51a560f652ab4ff9b0748a26425b4d7a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:26.180902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb7164-27a0-4c97-8b23-58b044f1e9fe admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:26.182991 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f1bb7164-27a0-4c97-8b23-58b044f1e9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:26.196010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb7164-27a0-4c97-8b23-58b044f1e9fe admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:26.251648 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec6fba-e99c-4807-86da-81650eef78ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:26.252247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec6fba-e99c-4807-86da-81650eef78ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:26.252526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec6fba-e99c-4807-86da-81650eef78ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:26.252827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec6fba-e99c-4807-86da-81650eef78ca admin admin] PATH_INFO: `/v3/users/b918776128de40ec94d252b59e278651` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:26.253928 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec6fba-e99c-4807-86da-81650eef78ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b918776128de40ec94d252b59e278651)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:26.255823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec6fba-e99c-4807-86da-81650eef78ca admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:26.555102 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-99ec6fba-e99c-4807-86da-81650eef78ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b918776128de40ec94d252b59e278651'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:26.746205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-99ec6fba-e99c-4807-86da-81650eef78ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b918776128de40ec94d252b59e278651'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:26.746822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-99ec6fba-e99c-4807-86da-81650eef78ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b918776128de40ec94d252b59e278651'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:26.754480 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-99ec6fba-e99c-4807-86da-81650eef78ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b918776128de40ec94d252b59e278651'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:26.767875 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-99ec6fba-e99c-4807-86da-81650eef78ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b918776128de40ec94d252b59e278651'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:26.774246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1204/2414] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:02:26 2022] DELETE /identity/v3/users/b918776128de40ec94d252b59e278651 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:27.137106 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:27.138854 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:27.153999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b861e7-97fe-4f64-9a3a-da7377b5139e admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:27.212843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a55fba-655a-47ba-9561-476fc57a93ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:27.213610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a55fba-655a-47ba-9561-476fc57a93ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:27.213891 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a55fba-655a-47ba-9561-476fc57a93ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:27.214164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a55fba-655a-47ba-9561-476fc57a93ce admin admin] PATH_INFO: `/v3/projects/1225d5b2b1574d59afb9a614fb0488bc` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:27.215338 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a55fba-655a-47ba-9561-476fc57a93ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=1225d5b2b1574d59afb9a614fb0488bc)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:27.217450 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a55fba-655a-47ba-9561-476fc57a93ce admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:27.336931 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:27.340128 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b8a55fba-655a-47ba-9561-476fc57a93ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1225d5b2b1574d59afb9a614fb0488bc', 'request_id': 'req-b8a55fba-655a-47ba-9561-476fc57a93ce'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:27.346378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b8a55fba-655a-47ba-9561-476fc57a93ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1225d5b2b1574d59afb9a614fb0488bc', 'request_id': 'req-b8a55fba-655a-47ba-9561-476fc57a93ce'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:27.351707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b8a55fba-655a-47ba-9561-476fc57a93ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1225d5b2b1574d59afb9a614fb0488bc', 'request_id': 'req-b8a55fba-655a-47ba-9561-476fc57a93ce'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:27.476854 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b8a55fba-655a-47ba-9561-476fc57a93ce admin admin] The token cache is being invalidate because project 1225d5b2b1574d59afb9a614fb0488bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:02:27.477076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b8a55fba-655a-47ba-9561-476fc57a93ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:27.477727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b8a55fba-655a-47ba-9561-476fc57a93ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:27.479895 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1211/2415] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:02:27 2022] DELETE /identity/v3/projects/1225d5b2b1574d59afb9a614fb0488bc => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:27.484913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd97d3-bd96-4690-bac2-2590ac0284a0 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:27.490405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd97d3-bd96-4690-bac2-2590ac0284a0 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:27.490405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd97d3-bd96-4690-bac2-2590ac0284a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:27.505954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-d3bd97d3-bd96-4690-bac2-2590ac0284a0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:27.507155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d3bd97d3-bd96-4690-bac2-2590ac0284a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:27.566941 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1205/2416] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:27.575135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c690ff5-b8c2-41b3-be8d-f76715703ba6 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:27.575501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c690ff5-b8c2-41b3-be8d-f76715703ba6 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:27.575784 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c690ff5-b8c2-41b3-be8d-f76715703ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:27.595702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-4c690ff5-b8c2-41b3-be8d-f76715703ba6 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:27.596966 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4c690ff5-b8c2-41b3-be8d-f76715703ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:27.605658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1212/2417] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:27.609862 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd97d3-bd96-4690-bac2-2590ac0284a0 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:27.616014 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:27.620722 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd97d3-bd96-4690-bac2-2590ac0284a0 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:27.625373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb77ee4-6740-4158-8a15-9ef267d876ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:27.633650 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb77ee4-6740-4158-8a15-9ef267d876ba admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:27.633650 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb77ee4-6740-4158-8a15-9ef267d876ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:27.633650 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb77ee4-6740-4158-8a15-9ef267d876ba admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:27.633650 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb77ee4-6740-4158-8a15-9ef267d876ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:02:27.633650 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb77ee4-6740-4158-8a15-9ef267d876ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:27.633650 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb77ee4-6740-4158-8a15-9ef267d876ba admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:27.640608 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1206/2418] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:02:27.644706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c690ff5-b8c2-41b3-be8d-f76715703ba6 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:27.653791 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c690ff5-b8c2-41b3-be8d-f76715703ba6 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:27.659039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdf673-f567-401f-a40e-1aeb2f2905fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:27.659518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf673-f567-401f-a40e-1aeb2f2905fb admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:27.659797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf673-f567-401f-a40e-1aeb2f2905fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:27.660067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf673-f567-401f-a40e-1aeb2f2905fb admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:27.660719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdf673-f567-401f-a40e-1aeb2f2905fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:27.662633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdf673-f567-401f-a40e-1aeb2f2905fb admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:27.772269 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1213/2419] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:02:27 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:27.777027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb77ee4-6740-4158-8a15-9ef267d876ba admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:27.791558 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb77ee4-6740-4158-8a15-9ef267d876ba admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:27.847839 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3d25f-62ce-4040-9d94-7bdd73e2c753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:27.848407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3d25f-62ce-4040-9d94-7bdd73e2c753 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:27.848706 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3d25f-62ce-4040-9d94-7bdd73e2c753 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:27.848985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3d25f-62ce-4040-9d94-7bdd73e2c753 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:27.849629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3d25f-62ce-4040-9d94-7bdd73e2c753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:27.851543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3d25f-62ce-4040-9d94-7bdd73e2c753 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:27.977522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1207/2420] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:02:27 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:27.980409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdf673-f567-401f-a40e-1aeb2f2905fb admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:27.990711 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdf673-f567-401f-a40e-1aeb2f2905fb admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:27.995697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ea29f-7515-4961-8a20-c65b999e7b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:27.996321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ea29f-7515-4961-8a20-c65b999e7b67 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:27.996598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ea29f-7515-4961-8a20-c65b999e7b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:27.996902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ea29f-7515-4961-8a20-c65b999e7b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:27.997539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ea29f-7515-4961-8a20-c65b999e7b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:27.999118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ea29f-7515-4961-8a20-c65b999e7b67 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:28.009002 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1214/2421] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:02:28.013312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3d25f-62ce-4040-9d94-7bdd73e2c753 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:28.023643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3d25f-62ce-4040-9d94-7bdd73e2c753 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:28.030733 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036ec27-f61a-40e4-96f6-7c5ac30ff14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:28.031290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036ec27-f61a-40e4-96f6-7c5ac30ff14e admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:28.032024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036ec27-f61a-40e4-96f6-7c5ac30ff14e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:28.032024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036ec27-f61a-40e4-96f6-7c5ac30ff14e admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:28.032720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036ec27-f61a-40e4-96f6-7c5ac30ff14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:28.034540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036ec27-f61a-40e4-96f6-7c5ac30ff14e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:28.047132 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1208/2422] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:28.051274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ea29f-7515-4961-8a20-c65b999e7b67 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:28.061813 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ea29f-7515-4961-8a20-c65b999e7b67 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:28.066594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83418a88-ed32-4cbe-8413-4cb52a1e6f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:28.067504 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83418a88-ed32-4cbe-8413-4cb52a1e6f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:28.067789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83418a88-ed32-4cbe-8413-4cb52a1e6f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:28.068059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83418a88-ed32-4cbe-8413-4cb52a1e6f4d admin admin] PATH_INFO: `/v3/projects/5cc4b04dfec94a0da738335594cd34c4/users/37677ac413c54527867788952967f886/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:28.084637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83418a88-ed32-4cbe-8413-4cb52a1e6f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc4b04dfec94a0da738335594cd34c4, user_id=37677ac413c54527867788952967f886, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:28.087949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83418a88-ed32-4cbe-8413-4cb52a1e6f4d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:28.156914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1215/2423] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:28 2022] PUT /identity/v3/projects/5cc4b04dfec94a0da738335594cd34c4/users/37677ac413c54527867788952967f886/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:28.163683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a07b42-1c68-42a5-a2e5-599a4a91ef4d None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:28.163968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a07b42-1c68-42a5-a2e5-599a4a91ef4d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:28.164262 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a07b42-1c68-42a5-a2e5-599a4a91ef4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:28.195191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-30a07b42-1c68-42a5-a2e5-599a4a91ef4d None None] MFA Rules not processed for user `37677ac413c54527867788952967f886`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:28.196522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-30a07b42-1c68-42a5-a2e5-599a4a91ef4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:28.262803 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1209/2424] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:02:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:28.267440 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83418a88-ed32-4cbe-8413-4cb52a1e6f4d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:28.328209 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83418a88-ed32-4cbe-8413-4cb52a1e6f4d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:28.332863 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed47ca-7f36-41e4-b643-f71abad4f263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:28.333460 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed47ca-7f36-41e4-b643-f71abad4f263 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:28.333738 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed47ca-7f36-41e4-b643-f71abad4f263 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:28.334009 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed47ca-7f36-41e4-b643-f71abad4f263 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:28.334625 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed47ca-7f36-41e4-b643-f71abad4f263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:28.336503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed47ca-7f36-41e4-b643-f71abad4f263 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:28.415570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1216/2425] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:02:28 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:28.419852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a07b42-1c68-42a5-a2e5-599a4a91ef4d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:28.430600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a07b42-1c68-42a5-a2e5-599a4a91ef4d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:28.487209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bcaf0-35e8-4850-bb2f-a923d05ab6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:28.487873 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bcaf0-35e8-4850-bb2f-a923d05ab6d3 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:28.488130 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bcaf0-35e8-4850-bb2f-a923d05ab6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:28.488405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bcaf0-35e8-4850-bb2f-a923d05ab6d3 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:28.489059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bcaf0-35e8-4850-bb2f-a923d05ab6d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:28.491007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bcaf0-35e8-4850-bb2f-a923d05ab6d3 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:28.499561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:28.592449 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1210/2426] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:02:28 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:28.596521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed47ca-7f36-41e4-b643-f71abad4f263 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:28.608989 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed47ca-7f36-41e4-b643-f71abad4f263 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:28.613622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8ca2e-6cc2-4c84-bda0-772751ef9010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:28.614220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8ca2e-6cc2-4c84-bda0-772751ef9010 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:28.614501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8ca2e-6cc2-4c84-bda0-772751ef9010 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:28.614776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8ca2e-6cc2-4c84-bda0-772751ef9010 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:28.615413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8ca2e-6cc2-4c84-bda0-772751ef9010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:28.617049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8ca2e-6cc2-4c84-bda0-772751ef9010 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:28.628490 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1217/2427] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:28.632758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bcaf0-35e8-4850-bb2f-a923d05ab6d3 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:28.643716 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bcaf0-35e8-4850-bb2f-a923d05ab6d3 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:28.648278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08efd1-ec06-484d-ad58-6f3ab4a569d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:28.648942 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08efd1-ec06-484d-ad58-6f3ab4a569d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:28.649222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08efd1-ec06-484d-ad58-6f3ab4a569d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:28.649494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08efd1-ec06-484d-ad58-6f3ab4a569d8 admin admin] PATH_INFO: `/v3/projects/e9190e82b50d42fe95bb15d6d9c57b90/users/d49e1a4a9d5a43289c99f1b577c2039c/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:28.663889 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08efd1-ec06-484d-ad58-6f3ab4a569d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9190e82b50d42fe95bb15d6d9c57b90, user_id=d49e1a4a9d5a43289c99f1b577c2039c, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:28.666940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08efd1-ec06-484d-ad58-6f3ab4a569d8 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:28.764470 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1211/2428] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:28 2022] PUT /identity/v3/projects/e9190e82b50d42fe95bb15d6d9c57b90/users/d49e1a4a9d5a43289c99f1b577c2039c/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:28.768476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8ca2e-6cc2-4c84-bda0-772751ef9010 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:28.778586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8ca2e-6cc2-4c84-bda0-772751ef9010 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:28.819583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:28.834897 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f23058-ba2d-4311-af61-66eae009955f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:28.836197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23058-ba2d-4311-af61-66eae009955f admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:28.836197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23058-ba2d-4311-af61-66eae009955f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:28.836197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23058-ba2d-4311-af61-66eae009955f admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:28.836637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f23058-ba2d-4311-af61-66eae009955f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:28.838606 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f23058-ba2d-4311-af61-66eae009955f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:28.848174 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1218/2429] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:28.852208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08efd1-ec06-484d-ad58-6f3ab4a569d8 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:28.862982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08efd1-ec06-484d-ad58-6f3ab4a569d8 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:28.867589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bfc42-40a2-4da3-a14b-f74366d8c20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:28.868227 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bfc42-40a2-4da3-a14b-f74366d8c20d admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:28.868552 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bfc42-40a2-4da3-a14b-f74366d8c20d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:28.869602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bfc42-40a2-4da3-a14b-f74366d8c20d admin admin] PATH_INFO: `/v3/projects/e9190e82b50d42fe95bb15d6d9c57b90/users/d49e1a4a9d5a43289c99f1b577c2039c/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:28.871535 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2bfc42-40a2-4da3-a14b-f74366d8c20d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9190e82b50d42fe95bb15d6d9c57b90, user_id=d49e1a4a9d5a43289c99f1b577c2039c, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:28.874177 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2bfc42-40a2-4da3-a14b-f74366d8c20d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:28.884155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1212/2430] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:28 2022] PUT /identity/v3/projects/e9190e82b50d42fe95bb15d6d9c57b90/users/d49e1a4a9d5a43289c99f1b577c2039c/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:28.895269 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff07759-8a47-48cf-b0c5-2b8f470c9d95 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:28.899361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff07759-8a47-48cf-b0c5-2b8f470c9d95 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:28.899361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff07759-8a47-48cf-b0c5-2b8f470c9d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:28.925221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-dff07759-8a47-48cf-b0c5-2b8f470c9d95 None None] MFA Rules not processed for user `d49e1a4a9d5a43289c99f1b577c2039c`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:28.926442 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-dff07759-8a47-48cf-b0c5-2b8f470c9d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:28.994372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1219/2431] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:02:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:38.880421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:INSERT=5,keystone:SELECT=17 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:38.990902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=43 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:03:16.059635 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bfc42-40a2-4da3-a14b-f74366d8c20d admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:16.061686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fe2bfc42-40a2-4da3-a14b-f74366d8c20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:16.144835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bfc42-40a2-4da3-a14b-f74366d8c20d admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:16.150386 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752d2e5-de2a-4068-bdb4-4d70dc5d7fce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:16.151127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752d2e5-de2a-4068-bdb4-4d70dc5d7fce service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:16.151484 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752d2e5-de2a-4068-bdb4-4d70dc5d7fce service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:16.152232 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752d2e5-de2a-4068-bdb4-4d70dc5d7fce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:16.155134 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4752d2e5-de2a-4068-bdb4-4d70dc5d7fce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:16.167494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752d2e5-de2a-4068-bdb4-4d70dc5d7fce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:16.169204 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752d2e5-de2a-4068-bdb4-4d70dc5d7fce service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:16.179960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1213/2432] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:03:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:03:16.245955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff07759-8a47-48cf-b0c5-2b8f470c9d95 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:16.251044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-dff07759-8a47-48cf-b0c5-2b8f470c9d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:16.314382 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff07759-8a47-48cf-b0c5-2b8f470c9d95 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:16.318540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f8db6-41b8-4cbc-b365-2e05ad68067a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:16.319155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f8db6-41b8-4cbc-b365-2e05ad68067a service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:16.319441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f8db6-41b8-4cbc-b365-2e05ad68067a service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:16.319720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f8db6-41b8-4cbc-b365-2e05ad68067a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:16.321499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-588f8db6-41b8-4cbc-b365-2e05ad68067a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:16.332176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f8db6-41b8-4cbc-b365-2e05ad68067a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:16.333644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f8db6-41b8-4cbc-b365-2e05ad68067a service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:16.342804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1220/2433] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:03:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:03:26.175473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=34 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:03:26.338850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=26 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:03:57.869479 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752d2e5-de2a-4068-bdb4-4d70dc5d7fce service nova] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:57.870979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4752d2e5-de2a-4068-bdb4-4d70dc5d7fce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:57.885579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752d2e5-de2a-4068-bdb4-4d70dc5d7fce service nova] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:57.891084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f85c10-5029-4331-b80c-cfbcffd8b953 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:57.891697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f85c10-5029-4331-b80c-cfbcffd8b953 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:57.891980 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f85c10-5029-4331-b80c-cfbcffd8b953 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:57.892254 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f85c10-5029-4331-b80c-cfbcffd8b953 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:57.895008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-57f85c10-5029-4331-b80c-cfbcffd8b953 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:57.974933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f85c10-5029-4331-b80c-cfbcffd8b953 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:57.976528 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f85c10-5029-4331-b80c-cfbcffd8b953 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:57.994556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1214/2434] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:03:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:04:07.991973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=41 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:04:09.283546 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f8db6-41b8-4cbc-b365-2e05ad68067a service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:04:09.296585 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f8db6-41b8-4cbc-b365-2e05ad68067a service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:04:09.302206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be943963-98eb-4984-853a-cfe743369a6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:04:09.302795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be943963-98eb-4984-853a-cfe743369a6c service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:04:09.303075 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be943963-98eb-4984-853a-cfe743369a6c service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:04:09.303348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be943963-98eb-4984-853a-cfe743369a6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:04:09.305074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-be943963-98eb-4984-853a-cfe743369a6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:04:09.383403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be943963-98eb-4984-853a-cfe743369a6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:04:09.385003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be943963-98eb-4984-853a-cfe743369a6c service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:04:09.403115 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1221/2435] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:04:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:04:19.348415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f85c10-5029-4331-b80c-cfbcffd8b953 service nova] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:04:19.349913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-57f85c10-5029-4331-b80c-cfbcffd8b953 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:04:19.400574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=41 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:04:19.408318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f85c10-5029-4331-b80c-cfbcffd8b953 service nova] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:04:19.412003 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89596276-50c2-4396-8855-ad16690f59a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:04:19.412608 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89596276-50c2-4396-8855-ad16690f59a1 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:04:19.423970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89596276-50c2-4396-8855-ad16690f59a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:04:19.423970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89596276-50c2-4396-8855-ad16690f59a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:04:19.423970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89596276-50c2-4396-8855-ad16690f59a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:04:19.423970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89596276-50c2-4396-8855-ad16690f59a1 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:04:19.433824 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1215/2436] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:04:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:04:29.429623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=26 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:05:13.321854 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be943963-98eb-4984-853a-cfe743369a6c service nova] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:05:13.323399 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-be943963-98eb-4984-853a-cfe743369a6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:05:13.391144 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be943963-98eb-4984-853a-cfe743369a6c service nova] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:05:13.396144 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40880d-2fee-414b-b7bd-5571731aa6a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:05:13.396786 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40880d-2fee-414b-b7bd-5571731aa6a8 service placement] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:05:13.397070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40880d-2fee-414b-b7bd-5571731aa6a8 service placement] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:05:13.397346 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40880d-2fee-414b-b7bd-5571731aa6a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:05:13.399042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ee40880d-2fee-414b-b7bd-5571731aa6a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:05:13.410695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40880d-2fee-414b-b7bd-5571731aa6a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:05:13.412272 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40880d-2fee-414b-b7bd-5571731aa6a8 service placement] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:05:13.422770 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1222/2437] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:05:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:05:23.418807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=32 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:08:19.042580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89596276-50c2-4396-8855-ad16690f59a1 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:19.051956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89596276-50c2-4396-8855-ad16690f59a1 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:19.055696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332f759-c37a-4d4e-976f-8a92725ad7e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:19.056297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332f759-c37a-4d4e-976f-8a92725ad7e0 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:19.056580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332f759-c37a-4d4e-976f-8a92725ad7e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:19.056883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332f759-c37a-4d4e-976f-8a92725ad7e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:19.064945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332f759-c37a-4d4e-976f-8a92725ad7e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:19.069741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332f759-c37a-4d4e-976f-8a92725ad7e0 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:19.074439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1216/2438] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:08:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:08:19.820874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40880d-2fee-414b-b7bd-5571731aa6a8 service placement] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:19.822273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ee40880d-2fee-414b-b7bd-5571731aa6a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:19.835120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332f759-c37a-4d4e-976f-8a92725ad7e0 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:19.835270 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40880d-2fee-414b-b7bd-5571731aa6a8 service placement] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:19.839038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762de0f-49a5-47d6-8dea-d105fe2bf703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:19.839646 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762de0f-49a5-47d6-8dea-d105fe2bf703 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:19.839931 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762de0f-49a5-47d6-8dea-d105fe2bf703 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:19.840211 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762de0f-49a5-47d6-8dea-d105fe2bf703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:19.842705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0762de0f-49a5-47d6-8dea-d105fe2bf703 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:19.845588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332f759-c37a-4d4e-976f-8a92725ad7e0 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:19.849382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c880f6-7b9a-45b1-b562-181e37f732dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:19.849978 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c880f6-7b9a-45b1-b562-181e37f732dd service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:19.850259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c880f6-7b9a-45b1-b562-181e37f732dd service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:19.850537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c880f6-7b9a-45b1-b562-181e37f732dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:19.852232 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-90c880f6-7b9a-45b1-b562-181e37f732dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:19.856817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762de0f-49a5-47d6-8dea-d105fe2bf703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:19.858471 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762de0f-49a5-47d6-8dea-d105fe2bf703 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:19.866505 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c880f6-7b9a-45b1-b562-181e37f732dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:19.867876 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c880f6-7b9a-45b1-b562-181e37f732dd service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:19.869892 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1223/2439] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:08:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:08:19.877504 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762de0f-49a5-47d6-8dea-d105fe2bf703 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:19.879651 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1217/2440] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:08:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:08:19.886979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762de0f-49a5-47d6-8dea-d105fe2bf703 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:19.889507 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c880f6-7b9a-45b1-b562-181e37f732dd service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:19.892037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4c02b-5002-422e-a24c-0b52a5a93224 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:19.892676 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4c02b-5002-422e-a24c-0b52a5a93224 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:19.892968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4c02b-5002-422e-a24c-0b52a5a93224 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:19.893244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4c02b-5002-422e-a24c-0b52a5a93224 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:19.895339 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-18f4c02b-5002-422e-a24c-0b52a5a93224 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:19.899701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c880f6-7b9a-45b1-b562-181e37f732dd service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:19.904536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638e8a5-0625-43f6-83d8-5cab3eabc0c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:19.905464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638e8a5-0625-43f6-83d8-5cab3eabc0c4 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:19.905818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638e8a5-0625-43f6-83d8-5cab3eabc0c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:19.906173 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638e8a5-0625-43f6-83d8-5cab3eabc0c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:19.908176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e638e8a5-0625-43f6-83d8-5cab3eabc0c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:19.994440 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638e8a5-0625-43f6-83d8-5cab3eabc0c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:19.994681 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4c02b-5002-422e-a24c-0b52a5a93224 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:19.995999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638e8a5-0625-43f6-83d8-5cab3eabc0c4 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:19.996140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4c02b-5002-422e-a24c-0b52a5a93224 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:20.015097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1218/2441] 10.222.0.26 () {60 vars in 1396 bytes} [Wed Aug 24 08:08:19 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:08:20.016773 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1224/2442] 10.222.0.26 () {60 vars in 1396 bytes} [Wed Aug 24 08:08:19 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:08:20.187281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638e8a5-0625-43f6-83d8-5cab3eabc0c4 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:20.198012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638e8a5-0625-43f6-83d8-5cab3eabc0c4 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:20.198152 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4c02b-5002-422e-a24c-0b52a5a93224 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:20.203522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9ca85-5d4b-4726-8b8a-d94e137c9db7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:20.204137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9ca85-5d4b-4726-8b8a-d94e137c9db7 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:20.204416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9ca85-5d4b-4726-8b8a-d94e137c9db7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:20.204717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9ca85-5d4b-4726-8b8a-d94e137c9db7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:20.206536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4ff9ca85-5d4b-4726-8b8a-d94e137c9db7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:20.211888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4c02b-5002-422e-a24c-0b52a5a93224 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:20.217017 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a9610-1cb8-4fea-86bd-533cbeb3567d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:20.218205 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a9610-1cb8-4fea-86bd-533cbeb3567d service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:20.218765 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a9610-1cb8-4fea-86bd-533cbeb3567d service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:20.219225 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a9610-1cb8-4fea-86bd-533cbeb3567d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:20.225636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-289a9610-1cb8-4fea-86bd-533cbeb3567d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:20.336898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9ca85-5d4b-4726-8b8a-d94e137c9db7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:20.340271 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9ca85-5d4b-4726-8b8a-d94e137c9db7 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:20.358145 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1219/2443] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:08:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:08:20.372906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a9610-1cb8-4fea-86bd-533cbeb3567d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:20.374854 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a9610-1cb8-4fea-86bd-533cbeb3567d service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:20.415846 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1225/2444] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:08:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:08:20.468795 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9ca85-5d4b-4726-8b8a-d94e137c9db7 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:20.518644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a9610-1cb8-4fea-86bd-533cbeb3567d service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:20.538112 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a9610-1cb8-4fea-86bd-533cbeb3567d service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:20.538625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:08:20.547201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85812db-b138-4ccb-ae99-c76b0008376e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:20.549363 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85812db-b138-4ccb-ae99-c76b0008376e service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:20.550396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85812db-b138-4ccb-ae99-c76b0008376e service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:20.550890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85812db-b138-4ccb-ae99-c76b0008376e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:20.555924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c85812db-b138-4ccb-ae99-c76b0008376e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:20.568047 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9ca85-5d4b-4726-8b8a-d94e137c9db7 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:20.573504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2050ec5-c1ba-4caa-8d75-9aad11d8d643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:20.575856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2050ec5-c1ba-4caa-8d75-9aad11d8d643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:20.576257 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2050ec5-c1ba-4caa-8d75-9aad11d8d643 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:20.576614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2050ec5-c1ba-4caa-8d75-9aad11d8d643 admin admin] PATH_INFO: `/v3/users/37677ac413c54527867788952967f886` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:20.577934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2050ec5-c1ba-4caa-8d75-9aad11d8d643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37677ac413c54527867788952967f886)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:20.580365 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2050ec5-c1ba-4caa-8d75-9aad11d8d643 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:20.673845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:08:20.689372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85812db-b138-4ccb-ae99-c76b0008376e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:20.689372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85812db-b138-4ccb-ae99-c76b0008376e service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:20.695570 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1226/2445] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:08:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:08:20.803950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d2050ec5-c1ba-4caa-8d75-9aad11d8d643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37677ac413c54527867788952967f886'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:20.905310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d2050ec5-c1ba-4caa-8d75-9aad11d8d643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37677ac413c54527867788952967f886'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:20.905310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d2050ec5-c1ba-4caa-8d75-9aad11d8d643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37677ac413c54527867788952967f886'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:20.905310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d2050ec5-c1ba-4caa-8d75-9aad11d8d643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37677ac413c54527867788952967f886'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:20.916224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-d2050ec5-c1ba-4caa-8d75-9aad11d8d643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37677ac413c54527867788952967f886'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:20.929175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1220/2446] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:08:20 2022] DELETE /identity/v3/users/37677ac413c54527867788952967f886 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:08:20.929778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed6307-21c4-42c0-b6b2-e468f66cfc8c None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:20.929778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed6307-21c4-42c0-b6b2-e468f66cfc8c None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:20.929778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed6307-21c4-42c0-b6b2-e468f66cfc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:20.954579 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-eeed6307-21c4-42c0-b6b2-e468f66cfc8c None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:08:20.956021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-eeed6307-21c4-42c0-b6b2-e468f66cfc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:21.072509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1227/2447] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:08:21.084322 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2050ec5-c1ba-4caa-8d75-9aad11d8d643 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:21.085947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d2050ec5-c1ba-4caa-8d75-9aad11d8d643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:21.192417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2050ec5-c1ba-4caa-8d75-9aad11d8d643 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:21.197231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102f1a1-2a9c-4c09-9b93-39d169b53d78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:21.197837 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102f1a1-2a9c-4c09-9b93-39d169b53d78 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:21.198308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102f1a1-2a9c-4c09-9b93-39d169b53d78 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:21.198582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102f1a1-2a9c-4c09-9b93-39d169b53d78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:21.209167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102f1a1-2a9c-4c09-9b93-39d169b53d78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:21.213517 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102f1a1-2a9c-4c09-9b93-39d169b53d78 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:21.224831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1221/2448] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:08:21.432179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed6307-21c4-42c0-b6b2-e468f66cfc8c None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:21.434475 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-eeed6307-21c4-42c0-b6b2-e468f66cfc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:21.557821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed6307-21c4-42c0-b6b2-e468f66cfc8c None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:21.579603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c7460-c024-44fb-b1c9-d64ca8f64551 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:21.579603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c7460-c024-44fb-b1c9-d64ca8f64551 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:21.579603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c7460-c024-44fb-b1c9-d64ca8f64551 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:21.579603 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c7460-c024-44fb-b1c9-d64ca8f64551 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:21.580207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7c3c7460-c024-44fb-b1c9-d64ca8f64551 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:21.587696 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c7460-c024-44fb-b1c9-d64ca8f64551 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:21.587696 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c7460-c024-44fb-b1c9-d64ca8f64551 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:21.599959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1228/2449] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:08:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:08:21.725620 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102f1a1-2a9c-4c09-9b93-39d169b53d78 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:21.727081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6102f1a1-2a9c-4c09-9b93-39d169b53d78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:21.740842 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102f1a1-2a9c-4c09-9b93-39d169b53d78 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:21.746900 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:21.747585 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:21.748474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:21.748474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] PATH_INFO: `/v3/projects/5cc4b04dfec94a0da738335594cd34c4` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:21.749460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc4b04dfec94a0da738335594cd34c4)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:21.751361 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:21.913034 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc4b04dfec94a0da738335594cd34c4', 'request_id': 'req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:21.919257 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc4b04dfec94a0da738335594cd34c4', 'request_id': 'req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:21.922953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:08:22.037871 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc4b04dfec94a0da738335594cd34c4', 'request_id': 'req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:22.049083 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] The token cache is being invalidate because project 5cc4b04dfec94a0da738335594cd34c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:08:22.053015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:22.053015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:22.053015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1222/2450] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:08:21 2022] DELETE /identity/v3/projects/5cc4b04dfec94a0da738335594cd34c4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:08:22.056488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c7460-c024-44fb-b1c9-d64ca8f64551 service nova] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:22.067307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c7460-c024-44fb-b1c9-d64ca8f64551 service nova] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:22.123266 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91359b8a-6cdf-42c2-9a91-74e192b1a040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:22.123890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91359b8a-6cdf-42c2-9a91-74e192b1a040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:22.124170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91359b8a-6cdf-42c2-9a91-74e192b1a040 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:22.124443 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91359b8a-6cdf-42c2-9a91-74e192b1a040 admin admin] PATH_INFO: `/v3/users/d49e1a4a9d5a43289c99f1b577c2039c` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:22.125575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91359b8a-6cdf-42c2-9a91-74e192b1a040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d49e1a4a9d5a43289c99f1b577c2039c)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:22.127512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91359b8a-6cdf-42c2-9a91-74e192b1a040 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:22.149041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:08:22.296436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-91359b8a-6cdf-42c2-9a91-74e192b1a040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd49e1a4a9d5a43289c99f1b577c2039c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:22.303263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-91359b8a-6cdf-42c2-9a91-74e192b1a040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd49e1a4a9d5a43289c99f1b577c2039c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:22.357731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-91359b8a-6cdf-42c2-9a91-74e192b1a040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd49e1a4a9d5a43289c99f1b577c2039c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:22.358381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-91359b8a-6cdf-42c2-9a91-74e192b1a040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd49e1a4a9d5a43289c99f1b577c2039c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:22.370453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-91359b8a-6cdf-42c2-9a91-74e192b1a040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd49e1a4a9d5a43289c99f1b577c2039c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:22.376804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1229/2451] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:08:22 2022] DELETE /identity/v3/users/d49e1a4a9d5a43289c99f1b577c2039c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:08:22.793598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:22.795708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:22.813481 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e56973-cb2c-40b6-9965-0c4e5abe3c7e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:22.867265 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:22.867918 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:22.868200 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:22.868474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97 admin admin] PATH_INFO: `/v3/projects/e9190e82b50d42fe95bb15d6d9c57b90` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:22.869631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9190e82b50d42fe95bb15d6d9c57b90)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:22.871583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:22.969681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9190e82b50d42fe95bb15d6d9c57b90', 'request_id': 'req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:22.975748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9190e82b50d42fe95bb15d6d9c57b90', 'request_id': 'req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:23.066012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9190e82b50d42fe95bb15d6d9c57b90', 'request_id': 'req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:23.077320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97 admin admin] The token cache is being invalidate because project e9190e82b50d42fe95bb15d6d9c57b90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:08:23.077613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:23.078261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-9b752de7-6e5d-4ae1-96b4-b21f9feb0c97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:08:23.079750 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1223/2452] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:08:22 2022] DELETE /identity/v3/projects/e9190e82b50d42fe95bb15d6d9c57b90 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:08:23.084749 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76120a27-f840-469d-995d-7bc952da365d None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:23.085244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76120a27-f840-469d-995d-7bc952da365d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:23.085345 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76120a27-f840-469d-995d-7bc952da365d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:23.105465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-76120a27-f840-469d-995d-7bc952da365d None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:08:23.107299 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-76120a27-f840-469d-995d-7bc952da365d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:23.164705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1230/2453] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:08:23.173701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77519b8a-c243-4f2f-b0d5-87a76deca51b None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:23.173988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77519b8a-c243-4f2f-b0d5-87a76deca51b None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:23.174273 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77519b8a-c243-4f2f-b0d5-87a76deca51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:23.193770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-77519b8a-c243-4f2f-b0d5-87a76deca51b None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:08:23.194960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-77519b8a-c243-4f2f-b0d5-87a76deca51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:23.208127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1224/2454] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:08:23.212544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76120a27-f840-469d-995d-7bc952da365d None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:23.223181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76120a27-f840-469d-995d-7bc952da365d None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:23.227775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab0b0b-5366-4f90-afcd-4437586e5d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:23.228401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab0b0b-5366-4f90-afcd-4437586e5d2b admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:23.228708 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab0b0b-5366-4f90-afcd-4437586e5d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:23.228987 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab0b0b-5366-4f90-afcd-4437586e5d2b admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:23.229632 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab0b0b-5366-4f90-afcd-4437586e5d2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:08:23.229938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab0b0b-5366-4f90-afcd-4437586e5d2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:23.232323 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab0b0b-5366-4f90-afcd-4437586e5d2b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:23.243666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1231/2455] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:08:23.247994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77519b8a-c243-4f2f-b0d5-87a76deca51b None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:23.257067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77519b8a-c243-4f2f-b0d5-87a76deca51b None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:23.261729 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7e493-7421-44ae-b10c-5c009e4b0672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:23.262314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7e493-7421-44ae-b10c-5c009e4b0672 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:23.263162 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7e493-7421-44ae-b10c-5c009e4b0672 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:23.263576 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7e493-7421-44ae-b10c-5c009e4b0672 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:23.264211 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7e493-7421-44ae-b10c-5c009e4b0672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:23.266130 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7e493-7421-44ae-b10c-5c009e4b0672 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:23.381383 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1225/2456] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:08:23 2022] POST /identity/v3/projects => generated 393 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:08:23.385690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab0b0b-5366-4f90-afcd-4437586e5d2b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:23.396275 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab0b0b-5366-4f90-afcd-4437586e5d2b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:23.451305 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7b85a-7927-4033-b3d3-e64a17b6ad8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:23.451895 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7b85a-7927-4033-b3d3-e64a17b6ad8d admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:23.452178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7b85a-7927-4033-b3d3-e64a17b6ad8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:23.452453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7b85a-7927-4033-b3d3-e64a17b6ad8d admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:23.453115 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7b85a-7927-4033-b3d3-e64a17b6ad8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:23.455039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7b85a-7927-4033-b3d3-e64a17b6ad8d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:23.546763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1232/2457] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:08:23 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:08:23.550871 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7e493-7421-44ae-b10c-5c009e4b0672 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:23.561822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7e493-7421-44ae-b10c-5c009e4b0672 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:23.566515 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c80365-599f-46f4-8fc9-a042c30a3bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:23.567120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c80365-599f-46f4-8fc9-a042c30a3bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:23.567407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c80365-599f-46f4-8fc9-a042c30a3bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:23.567681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c80365-599f-46f4-8fc9-a042c30a3bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:23.568321 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c80365-599f-46f4-8fc9-a042c30a3bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:23.569941 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c80365-599f-46f4-8fc9-a042c30a3bd4 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:23.583657 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1226/2458] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:08:23.584987 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7b85a-7927-4033-b3d3-e64a17b6ad8d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:23.596080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7b85a-7927-4033-b3d3-e64a17b6ad8d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:23.600794 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76cc87-bc67-4e4e-8437-63d18c992414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:23.601392 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76cc87-bc67-4e4e-8437-63d18c992414 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:23.601671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76cc87-bc67-4e4e-8437-63d18c992414 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:23.601940 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76cc87-bc67-4e4e-8437-63d18c992414 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:23.602574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76cc87-bc67-4e4e-8437-63d18c992414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:23.604163 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76cc87-bc67-4e4e-8437-63d18c992414 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:23.615140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1233/2459] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:08:23.619289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c80365-599f-46f4-8fc9-a042c30a3bd4 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:23.629808 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c80365-599f-46f4-8fc9-a042c30a3bd4 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:23.634592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e51cc4-a343-4dda-b51d-4ed3f1622129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:23.635183 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e51cc4-a343-4dda-b51d-4ed3f1622129 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:23.635466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e51cc4-a343-4dda-b51d-4ed3f1622129 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:23.635745 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e51cc4-a343-4dda-b51d-4ed3f1622129 admin admin] PATH_INFO: `/v3/projects/54cd29aac383473baa105d8e4f90bab9/users/232f5c97717841c4b6bde2ac517df754/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:23.650535 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e51cc4-a343-4dda-b51d-4ed3f1622129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54cd29aac383473baa105d8e4f90bab9, user_id=232f5c97717841c4b6bde2ac517df754, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:23.653319 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e51cc4-a343-4dda-b51d-4ed3f1622129 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:23.723006 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1227/2460] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:08:23 2022] PUT /identity/v3/projects/54cd29aac383473baa105d8e4f90bab9/users/232f5c97717841c4b6bde2ac517df754/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:08:23.734379 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8a4e2-a133-4325-a9a3-50b4153c6ce8 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:23.734670 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8a4e2-a133-4325-a9a3-50b4153c6ce8 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:23.734945 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8a4e2-a133-4325-a9a3-50b4153c6ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:23.742937 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:08:23.764727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-21c8a4e2-a133-4325-a9a3-50b4153c6ce8 None None] MFA Rules not processed for user `232f5c97717841c4b6bde2ac517df754`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:08:23.765935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-21c8a4e2-a133-4325-a9a3-50b4153c6ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:23.830360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1234/2461] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:08:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:08:23.834890 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e51cc4-a343-4dda-b51d-4ed3f1622129 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:23.852752 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:08:23.902024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e51cc4-a343-4dda-b51d-4ed3f1622129 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:23.907007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dea03-cac4-44f9-9539-570766ed1697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:23.908244 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dea03-cac4-44f9-9539-570766ed1697 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:23.908244 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dea03-cac4-44f9-9539-570766ed1697 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:23.908244 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dea03-cac4-44f9-9539-570766ed1697 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:23.908893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8dea03-cac4-44f9-9539-570766ed1697 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:23.910818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8dea03-cac4-44f9-9539-570766ed1697 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:23.975515 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1228/2462] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:08:23 2022] POST /identity/v3/projects => generated 389 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:08:23.979754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8a4e2-a133-4325-a9a3-50b4153c6ce8 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:23.990618 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8a4e2-a133-4325-a9a3-50b4153c6ce8 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:24.047804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28dda7-54ad-4041-84a9-46e7840ee547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:24.048721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28dda7-54ad-4041-84a9-46e7840ee547 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:24.049122 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28dda7-54ad-4041-84a9-46e7840ee547 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:24.049403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28dda7-54ad-4041-84a9-46e7840ee547 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:24.050024 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28dda7-54ad-4041-84a9-46e7840ee547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:24.051953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28dda7-54ad-4041-84a9-46e7840ee547 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:24.117502 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1235/2463] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:08:23 2022] POST /identity/v3/users => generated 301 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:08:24.121699 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dea03-cac4-44f9-9539-570766ed1697 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:24.132549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dea03-cac4-44f9-9539-570766ed1697 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:24.137172 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4841ffd-54ae-41af-8226-84c008c611da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:24.138011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4841ffd-54ae-41af-8226-84c008c611da admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:24.138286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4841ffd-54ae-41af-8226-84c008c611da admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:24.138563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4841ffd-54ae-41af-8226-84c008c611da admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:24.139212 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4841ffd-54ae-41af-8226-84c008c611da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:24.141015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4841ffd-54ae-41af-8226-84c008c611da admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:24.151819 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1229/2464] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:08:24.159249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28dda7-54ad-4041-84a9-46e7840ee547 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:24.170057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28dda7-54ad-4041-84a9-46e7840ee547 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:24.174712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f42f26-fe03-4f8f-8a53-b2fa7b1b3aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:24.175347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f42f26-fe03-4f8f-8a53-b2fa7b1b3aa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:24.175625 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f42f26-fe03-4f8f-8a53-b2fa7b1b3aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:24.175896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f42f26-fe03-4f8f-8a53-b2fa7b1b3aa5 admin admin] PATH_INFO: `/v3/projects/f0cd72652c954d4ebf3aa00259f4445e/users/d8e701158e66449c846aac6b67331cf6/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:24.191572 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f42f26-fe03-4f8f-8a53-b2fa7b1b3aa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0cd72652c954d4ebf3aa00259f4445e, user_id=d8e701158e66449c846aac6b67331cf6, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:24.194255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f42f26-fe03-4f8f-8a53-b2fa7b1b3aa5 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:24.260795 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1236/2465] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:08:24 2022] PUT /identity/v3/projects/f0cd72652c954d4ebf3aa00259f4445e/users/d8e701158e66449c846aac6b67331cf6/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:08:24.264965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4841ffd-54ae-41af-8226-84c008c611da admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:24.274997 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4841ffd-54ae-41af-8226-84c008c611da admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:24.331675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769fdb5-e43f-45b0-9d6f-10d7230c9877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:24.332273 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769fdb5-e43f-45b0-9d6f-10d7230c9877 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:24.332554 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769fdb5-e43f-45b0-9d6f-10d7230c9877 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:24.332861 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769fdb5-e43f-45b0-9d6f-10d7230c9877 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:24.333562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769fdb5-e43f-45b0-9d6f-10d7230c9877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:24.335157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769fdb5-e43f-45b0-9d6f-10d7230c9877 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:24.344977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1230/2466] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:08:24.349105 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f42f26-fe03-4f8f-8a53-b2fa7b1b3aa5 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:24.359908 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f42f26-fe03-4f8f-8a53-b2fa7b1b3aa5 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:24.364479 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0168f-a455-4cbd-b95d-7b0e5da82b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:24.365157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0168f-a455-4cbd-b95d-7b0e5da82b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:24.365437 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0168f-a455-4cbd-b95d-7b0e5da82b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:24.365706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0168f-a455-4cbd-b95d-7b0e5da82b93 admin admin] PATH_INFO: `/v3/projects/f0cd72652c954d4ebf3aa00259f4445e/users/d8e701158e66449c846aac6b67331cf6/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:24.367609 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0168f-a455-4cbd-b95d-7b0e5da82b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0cd72652c954d4ebf3aa00259f4445e, user_id=d8e701158e66449c846aac6b67331cf6, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:24.370282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0168f-a455-4cbd-b95d-7b0e5da82b93 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:24.381036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1237/2467] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:08:24 2022] PUT /identity/v3/projects/f0cd72652c954d4ebf3aa00259f4445e/users/d8e701158e66449c846aac6b67331cf6/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:08:24.391373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff2ea4-663a-4fde-bb2c-da089a4876cd None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:24.391664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff2ea4-663a-4fde-bb2c-da089a4876cd None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:24.391945 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff2ea4-663a-4fde-bb2c-da089a4876cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:24.421236 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-38ff2ea4-663a-4fde-bb2c-da089a4876cd None None] MFA Rules not processed for user `d8e701158e66449c846aac6b67331cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:08:24.422404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-38ff2ea4-663a-4fde-bb2c-da089a4876cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:24.488855 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1231/2468] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:08:24 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:08:34.377409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:08:34.485807 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=95,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:08:57.876410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0168f-a455-4cbd-b95d-7b0e5da82b93 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:57.878233 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9fc0168f-a455-4cbd-b95d-7b0e5da82b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:57.951535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0168f-a455-4cbd-b95d-7b0e5da82b93 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:57.957126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37081a-bef9-40f3-8c24-214676d0fc0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:57.957725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37081a-bef9-40f3-8c24-214676d0fc0f service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:57.958011 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37081a-bef9-40f3-8c24-214676d0fc0f service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:57.958291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37081a-bef9-40f3-8c24-214676d0fc0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:57.960012 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6b37081a-bef9-40f3-8c24-214676d0fc0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:58.016634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37081a-bef9-40f3-8c24-214676d0fc0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:58.018245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37081a-bef9-40f3-8c24-214676d0fc0f service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:58.027042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1238/2469] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:08:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:09:08.022947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=56 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:09:17.287237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff2ea4-663a-4fde-bb2c-da089a4876cd None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:17.288717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-38ff2ea4-663a-4fde-bb2c-da089a4876cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:17.344987 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff2ea4-663a-4fde-bb2c-da089a4876cd None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:17.349113 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cce525-2a5b-4c01-9a4c-4f192549ffd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:17.349708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cce525-2a5b-4c01-9a4c-4f192549ffd7 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:17.349988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cce525-2a5b-4c01-9a4c-4f192549ffd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:17.350259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cce525-2a5b-4c01-9a4c-4f192549ffd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:17.351967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-29cce525-2a5b-4c01-9a4c-4f192549ffd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:17.414185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cce525-2a5b-4c01-9a4c-4f192549ffd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:17.415627 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cce525-2a5b-4c01-9a4c-4f192549ffd7 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:17.426086 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1232/2470] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:09:21.033162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37081a-bef9-40f3-8c24-214676d0fc0f service nova] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:21.034618 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6b37081a-bef9-40f3-8c24-214676d0fc0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:21.103966 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37081a-bef9-40f3-8c24-214676d0fc0f service nova] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:21.109154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a678852-1362-4786-a7b5-6f0ede497db8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:21.109807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a678852-1362-4786-a7b5-6f0ede497db8 service placement] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:21.110091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a678852-1362-4786-a7b5-6f0ede497db8 service placement] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:21.110367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a678852-1362-4786-a7b5-6f0ede497db8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:21.112355 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8a678852-1362-4786-a7b5-6f0ede497db8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:21.123674 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a678852-1362-4786-a7b5-6f0ede497db8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:21.125313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a678852-1362-4786-a7b5-6f0ede497db8 service placement] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:21.136385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1239/2471] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:09:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:09:24.531449 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cce525-2a5b-4c01-9a4c-4f192549ffd7 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:24.543835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cce525-2a5b-4c01-9a4c-4f192549ffd7 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:24.549639 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386598b-24a3-4976-b2a0-0018e2cc1565 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:24.550751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386598b-24a3-4976-b2a0-0018e2cc1565 service placement] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:24.551043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386598b-24a3-4976-b2a0-0018e2cc1565 service placement] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:24.551297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386598b-24a3-4976-b2a0-0018e2cc1565 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:24.553638 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f386598b-24a3-4976-b2a0-0018e2cc1565 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:24.568612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386598b-24a3-4976-b2a0-0018e2cc1565 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:24.570292 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386598b-24a3-4976-b2a0-0018e2cc1565 service placement] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:24.580421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1233/2472] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:09:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:09:24.652201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a678852-1362-4786-a7b5-6f0ede497db8 service placement] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:24.653681 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8a678852-1362-4786-a7b5-6f0ede497db8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:24.668775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a678852-1362-4786-a7b5-6f0ede497db8 service placement] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:24.673232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e601e32-0b0e-47d3-9ab9-ce90818fe54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:24.673849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e601e32-0b0e-47d3-9ab9-ce90818fe54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:24.674834 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e601e32-0b0e-47d3-9ab9-ce90818fe54e admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:24.675136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e601e32-0b0e-47d3-9ab9-ce90818fe54e admin admin] PATH_INFO: `/v3/users/7f3a2203128f46528a40a9f55b54b5b9` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:24.676280 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e601e32-0b0e-47d3-9ab9-ce90818fe54e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f3a2203128f46528a40a9f55b54b5b9)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:24.678229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e601e32-0b0e-47d3-9ab9-ce90818fe54e admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:24.861259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5e601e32-0b0e-47d3-9ab9-ce90818fe54e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f3a2203128f46528a40a9f55b54b5b9'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:24.868818 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5e601e32-0b0e-47d3-9ab9-ce90818fe54e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f3a2203128f46528a40a9f55b54b5b9'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:24.945608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5e601e32-0b0e-47d3-9ab9-ce90818fe54e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f3a2203128f46528a40a9f55b54b5b9'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:24.946168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5e601e32-0b0e-47d3-9ab9-ce90818fe54e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f3a2203128f46528a40a9f55b54b5b9'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:24.958321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5e601e32-0b0e-47d3-9ab9-ce90818fe54e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f3a2203128f46528a40a9f55b54b5b9'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:24.964124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1240/2473] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:09:24 2022] DELETE /identity/v3/users/7f3a2203128f46528a40a9f55b54b5b9 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:09:25.222646 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386598b-24a3-4976-b2a0-0018e2cc1565 service placement] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:25.233159 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386598b-24a3-4976-b2a0-0018e2cc1565 service placement] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:25.290290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:25.290921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:25.291202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:25.291476 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] PATH_INFO: `/v3/projects/7d8b8ee42f4d499499f24e29d7a8249d` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:25.292586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d8b8ee42f4d499499f24e29d7a8249d)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:25.294514 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:25.396464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:09:25.403166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d8b8ee42f4d499499f24e29d7a8249d', 'request_id': 'req-f054d011-19ef-43a8-8239-429db5533e50'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:25.409265 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d8b8ee42f4d499499f24e29d7a8249d', 'request_id': 'req-f054d011-19ef-43a8-8239-429db5533e50'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:25.459704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d8b8ee42f4d499499f24e29d7a8249d', 'request_id': 'req-f054d011-19ef-43a8-8239-429db5533e50'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:25.470343 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] The token cache is being invalidate because project 7d8b8ee42f4d499499f24e29d7a8249d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:09:25.470747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:25.471781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:25.473293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1234/2474] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:09:25 2022] DELETE /identity/v3/projects/7d8b8ee42f4d499499f24e29d7a8249d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:09:25.477417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e601e32-0b0e-47d3-9ab9-ce90818fe54e admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:25.478847 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5e601e32-0b0e-47d3-9ab9-ce90818fe54e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:25.491908 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e601e32-0b0e-47d3-9ab9-ce90818fe54e admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:25.548597 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91856551-c7f8-442b-8310-5e7b83dd6656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:25.549287 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91856551-c7f8-442b-8310-5e7b83dd6656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:25.549741 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91856551-c7f8-442b-8310-5e7b83dd6656 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:25.550024 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91856551-c7f8-442b-8310-5e7b83dd6656 admin admin] PATH_INFO: `/v3/users/dd89b9f028ca49bfaba4ad41a3545dd8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:25.552714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:09:25.563687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91856551-c7f8-442b-8310-5e7b83dd6656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd89b9f028ca49bfaba4ad41a3545dd8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:25.566351 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91856551-c7f8-442b-8310-5e7b83dd6656 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:25.759755 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-91856551-c7f8-442b-8310-5e7b83dd6656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd89b9f028ca49bfaba4ad41a3545dd8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:25.766562 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-91856551-c7f8-442b-8310-5e7b83dd6656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd89b9f028ca49bfaba4ad41a3545dd8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:25.843865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-91856551-c7f8-442b-8310-5e7b83dd6656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd89b9f028ca49bfaba4ad41a3545dd8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:25.844729 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-91856551-c7f8-442b-8310-5e7b83dd6656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd89b9f028ca49bfaba4ad41a3545dd8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:25.858752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-91856551-c7f8-442b-8310-5e7b83dd6656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd89b9f028ca49bfaba4ad41a3545dd8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:25.865697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1241/2475] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:09:25 2022] DELETE /identity/v3/users/dd89b9f028ca49bfaba4ad41a3545dd8 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:09:26.119621 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:26.121201 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:26.135715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054d011-19ef-43a8-8239-429db5533e50 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:26.194060 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:26.194692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:26.194997 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:26.195258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] PATH_INFO: `/v3/projects/036c09e638f64a5f8ec5c5b574473fb1` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:26.205660 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=036c09e638f64a5f8ec5c5b574473fb1)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:26.207585 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:26.311932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '036c09e638f64a5f8ec5c5b574473fb1', 'request_id': 'req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:26.318521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '036c09e638f64a5f8ec5c5b574473fb1', 'request_id': 'req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:26.442730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '036c09e638f64a5f8ec5c5b574473fb1', 'request_id': 'req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:26.453520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] The token cache is being invalidate because project 036c09e638f64a5f8ec5c5b574473fb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:09:26.453813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:26.454466 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:26.456014 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1235/2476] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:09:26 2022] DELETE /identity/v3/projects/036c09e638f64a5f8ec5c5b574473fb1 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:09:35.863557 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:09:36.450471 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=65,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:10:19.288286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91856551-c7f8-442b-8310-5e7b83dd6656 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:10:19.289740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-91856551-c7f8-442b-8310-5e7b83dd6656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:10:19.428701 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91856551-c7f8-442b-8310-5e7b83dd6656 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:10:19.486491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20632d4-d7d6-4c38-9513-2d183245b72b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:10:19.487099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20632d4-d7d6-4c38-9513-2d183245b72b service placement] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:10:19.487385 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20632d4-d7d6-4c38-9513-2d183245b72b service placement] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:10:19.487664 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20632d4-d7d6-4c38-9513-2d183245b72b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:10:19.490100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d20632d4-d7d6-4c38-9513-2d183245b72b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:10:19.503159 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20632d4-d7d6-4c38-9513-2d183245b72b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:10:19.504893 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20632d4-d7d6-4c38-9513-2d183245b72b service placement] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:10:19.514536 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1242/2477] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:10:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:10:21.544562 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:10:21.550136 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:10:21.608249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d474f2-cf75-41ba-add6-2aeba9e0a7a6 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:10:21.611880 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d37c0-9d08-4b24-8e6a-999cc7441543 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:10:21.612480 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d37c0-9d08-4b24-8e6a-999cc7441543 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:10:21.612801 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d37c0-9d08-4b24-8e6a-999cc7441543 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:10:21.613078 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d37c0-9d08-4b24-8e6a-999cc7441543 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:10:21.614845 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b42d37c0-9d08-4b24-8e6a-999cc7441543 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:10:21.671890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20632d4-d7d6-4c38-9513-2d183245b72b service placement] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:10:21.673314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d20632d4-d7d6-4c38-9513-2d183245b72b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:10:21.676300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d37c0-9d08-4b24-8e6a-999cc7441543 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:10:21.681403 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d37c0-9d08-4b24-8e6a-999cc7441543 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:10:21.685890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20632d4-d7d6-4c38-9513-2d183245b72b service placement] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:10:21.687934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1236/2478] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:10:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:10:21.689746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b80817-253c-4281-a55a-5ae7204783cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:10:21.690286 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b80817-253c-4281-a55a-5ae7204783cb service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:10:21.690569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b80817-253c-4281-a55a-5ae7204783cb service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:10:21.690841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b80817-253c-4281-a55a-5ae7204783cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:10:21.693438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-59b80817-253c-4281-a55a-5ae7204783cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:10:21.716992 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:10:21.762814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b80817-253c-4281-a55a-5ae7204783cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:10:21.764614 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b80817-253c-4281-a55a-5ae7204783cb service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:10:21.775651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1243/2479] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:10:21 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:10:31.683540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=50 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:10:31.770197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=19 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:21.959097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d37c0-9d08-4b24-8e6a-999cc7441543 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:21.976380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d37c0-9d08-4b24-8e6a-999cc7441543 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:21.979072 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e97820-047d-49cf-a775-20c3f282667b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:21.979678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e97820-047d-49cf-a775-20c3f282667b service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:21.979960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e97820-047d-49cf-a775-20c3f282667b service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:21.980238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e97820-047d-49cf-a775-20c3f282667b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:21.981999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-15e97820-047d-49cf-a775-20c3f282667b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:22.043925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e97820-047d-49cf-a775-20c3f282667b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:22.045664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e97820-047d-49cf-a775-20c3f282667b service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:22.055658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1237/2480] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:12:21 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:12:32.051305 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=32 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:13:26.689387 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b80817-253c-4281-a55a-5ae7204783cb service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:13:26.690800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-59b80817-253c-4281-a55a-5ae7204783cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:13:26.766700 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b80817-253c-4281-a55a-5ae7204783cb service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:13:26.770354 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb9ff4-d0f5-4dcb-af43-6c80f3285b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:13:26.770956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb9ff4-d0f5-4dcb-af43-6c80f3285b30 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:13:26.771238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb9ff4-d0f5-4dcb-af43-6c80f3285b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:13:26.771516 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb9ff4-d0f5-4dcb-af43-6c80f3285b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:13:26.773273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0ddb9ff4-d0f5-4dcb-af43-6c80f3285b30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:13:26.783137 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb9ff4-d0f5-4dcb-af43-6c80f3285b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:13:26.784487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb9ff4-d0f5-4dcb-af43-6c80f3285b30 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:13:26.792192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1244/2481] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:13:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:13:36.788808 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=35 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:14:22.851057 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e97820-047d-49cf-a775-20c3f282667b service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:22.859675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e97820-047d-49cf-a775-20c3f282667b service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:22.864030 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8ca46-c6dc-443e-b872-66ce67e0838b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:22.869771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8ca46-c6dc-443e-b872-66ce67e0838b service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:22.869771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8ca46-c6dc-443e-b872-66ce67e0838b service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:22.869771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8ca46-c6dc-443e-b872-66ce67e0838b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:22.869771 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2da8ca46-c6dc-443e-b872-66ce67e0838b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:22.948763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8ca46-c6dc-443e-b872-66ce67e0838b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:22.950258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8ca46-c6dc-443e-b872-66ce67e0838b service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:22.952111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb9ff4-d0f5-4dcb-af43-6c80f3285b30 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:22.961841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb9ff4-d0f5-4dcb-af43-6c80f3285b30 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:22.965455 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a9cca-28d9-439d-a7c0-0b4d329590ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:22.966041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a9cca-28d9-439d-a7c0-0b4d329590ce service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:22.966323 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a9cca-28d9-439d-a7c0-0b4d329590ce service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:22.966587 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a9cca-28d9-439d-a7c0-0b4d329590ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:22.968291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-277a9cca-28d9-439d-a7c0-0b4d329590ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:22.969484 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1238/2482] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:14:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:14:23.013435 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8ca46-c6dc-443e-b872-66ce67e0838b service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:23.022351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8ca46-c6dc-443e-b872-66ce67e0838b service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:23.026196 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b101228-225a-46ed-9b04-cdd6f0dfef0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:23.026794 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b101228-225a-46ed-9b04-cdd6f0dfef0e service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:23.027073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b101228-225a-46ed-9b04-cdd6f0dfef0e service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:23.027344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b101228-225a-46ed-9b04-cdd6f0dfef0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:23.029107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9b101228-225a-46ed-9b04-cdd6f0dfef0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:23.051694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a9cca-28d9-439d-a7c0-0b4d329590ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:23.053104 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a9cca-28d9-439d-a7c0-0b4d329590ce service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:23.072236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1245/2483] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:14:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:14:23.096838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b101228-225a-46ed-9b04-cdd6f0dfef0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:23.099359 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b101228-225a-46ed-9b04-cdd6f0dfef0e service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:23.110414 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1239/2484] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:14:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:14:23.511824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a9cca-28d9-439d-a7c0-0b4d329590ce service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:23.512187 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b101228-225a-46ed-9b04-cdd6f0dfef0e service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:23.513493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-277a9cca-28d9-439d-a7c0-0b4d329590ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:23.520905 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b101228-225a-46ed-9b04-cdd6f0dfef0e service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:23.524881 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871970f6-7a7a-48f3-8ab9-2af03c384e47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:23.525567 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871970f6-7a7a-48f3-8ab9-2af03c384e47 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:23.525917 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871970f6-7a7a-48f3-8ab9-2af03c384e47 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:23.526290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871970f6-7a7a-48f3-8ab9-2af03c384e47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:23.530041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-871970f6-7a7a-48f3-8ab9-2af03c384e47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application [None req-871970f6-7a7a-48f3-8ab9-2af03c384e47 service cinder] Could not find project: 7d8b8ee42f4d499499f24e29d7a8249d.: keystone.exception.ProjectNotFound: Could not find project: 7d8b8ee42f4d499499f24e29d7a8249d. Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 24 08:14:23.544101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:14:23.545645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:14:23.547023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application with Lock( Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:14:23.548399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:14:23.549813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 7d8b8ee42f4d499499f24e29d7a8249d. Aug 24 08:14:23.549813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: ERROR keystone.server.flask.application Aug 24 08:14:23.549813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1240/2485] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:14:23 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 34 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 24 08:14:23.627731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a9cca-28d9-439d-a7c0-0b4d329590ce service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:23.634337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3e1bf-ac1c-4c46-a0cd-ea53f8e3eee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:23.634337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3e1bf-ac1c-4c46-a0cd-ea53f8e3eee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:23.634337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3e1bf-ac1c-4c46-a0cd-ea53f8e3eee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:23.634337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3e1bf-ac1c-4c46-a0cd-ea53f8e3eee7 admin admin] PATH_INFO: `/v3/users/232f5c97717841c4b6bde2ac517df754` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:23.635097 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3e1bf-ac1c-4c46-a0cd-ea53f8e3eee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=232f5c97717841c4b6bde2ac517df754)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:23.637052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3e1bf-ac1c-4c46-a0cd-ea53f8e3eee7 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:23.893424 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871970f6-7a7a-48f3-8ab9-2af03c384e47 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:23.904237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871970f6-7a7a-48f3-8ab9-2af03c384e47 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:23.909534 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:14:23.909683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5028b4b-1839-4103-a2c1-d1c28e09ec88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:23.909683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5028b4b-1839-4103-a2c1-d1c28e09ec88 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:23.909683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5028b4b-1839-4103-a2c1-d1c28e09ec88 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:23.909683 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5028b4b-1839-4103-a2c1-d1c28e09ec88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:23.911559 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b5a3e1bf-ac1c-4c46-a0cd-ea53f8e3eee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '232f5c97717841c4b6bde2ac517df754'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:23.912178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e5028b4b-1839-4103-a2c1-d1c28e09ec88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:23.918629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b5a3e1bf-ac1c-4c46-a0cd-ea53f8e3eee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '232f5c97717841c4b6bde2ac517df754'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:23.970385 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:14:23.998101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b5a3e1bf-ac1c-4c46-a0cd-ea53f8e3eee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '232f5c97717841c4b6bde2ac517df754'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:23.998924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b5a3e1bf-ac1c-4c46-a0cd-ea53f8e3eee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '232f5c97717841c4b6bde2ac517df754'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:24.001325 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5028b4b-1839-4103-a2c1-d1c28e09ec88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:24.002769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5028b4b-1839-4103-a2c1-d1c28e09ec88 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:24.012440 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b5a3e1bf-ac1c-4c46-a0cd-ea53f8e3eee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '232f5c97717841c4b6bde2ac517df754'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:24.019366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1246/2486] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:14:23 2022] DELETE /identity/v3/users/232f5c97717841c4b6bde2ac517df754 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:14:24.024262 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ae3ba-5542-429d-a90e-37c04ded8dc6 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:24.024434 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1241/2487] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:14:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:14:24.024743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ae3ba-5542-429d-a90e-37c04ded8dc6 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:24.025028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ae3ba-5542-429d-a90e-37c04ded8dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:24.055278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-1c0ae3ba-5542-429d-a90e-37c04ded8dc6 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:14:24.057018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-1c0ae3ba-5542-429d-a90e-37c04ded8dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:24.121853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1247/2488] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:14:24.128277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5028b4b-1839-4103-a2c1-d1c28e09ec88 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:24.130279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e5028b4b-1839-4103-a2c1-d1c28e09ec88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:24.141265 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5028b4b-1839-4103-a2c1-d1c28e09ec88 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:24.193758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b610b3-4db7-4a1e-af77-6a73880a3f0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:24.194352 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b610b3-4db7-4a1e-af77-6a73880a3f0c service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:24.194630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b610b3-4db7-4a1e-af77-6a73880a3f0c service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:24.194929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b610b3-4db7-4a1e-af77-6a73880a3f0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:24.203785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b610b3-4db7-4a1e-af77-6a73880a3f0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:24.205268 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b610b3-4db7-4a1e-af77-6a73880a3f0c service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:24.215260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1242/2489] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:14:24.653505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ae3ba-5542-429d-a90e-37c04ded8dc6 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:24.655033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-1c0ae3ba-5542-429d-a90e-37c04ded8dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:24.667678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ae3ba-5542-429d-a90e-37c04ded8dc6 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:24.674007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:24.674007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:24.674007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:24.674007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] PATH_INFO: `/v3/projects/54cd29aac383473baa105d8e4f90bab9` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:24.675303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54cd29aac383473baa105d8e4f90bab9)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:24.677881 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:24.818641 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54cd29aac383473baa105d8e4f90bab9', 'request_id': 'req-f6810a8a-c598-423b-ab37-6c500ddd1585'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:24.824733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54cd29aac383473baa105d8e4f90bab9', 'request_id': 'req-f6810a8a-c598-423b-ab37-6c500ddd1585'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:24.831445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54cd29aac383473baa105d8e4f90bab9', 'request_id': 'req-f6810a8a-c598-423b-ab37-6c500ddd1585'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:24.931789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] The token cache is being invalidate because project 54cd29aac383473baa105d8e4f90bab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:14:24.932012 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:24.932693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:24.934196 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1248/2490] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:14:24 2022] DELETE /identity/v3/projects/54cd29aac383473baa105d8e4f90bab9 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:14:24.938214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b610b3-4db7-4a1e-af77-6a73880a3f0c service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:24.947819 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b610b3-4db7-4a1e-af77-6a73880a3f0c service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:25.002949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf6d10-542f-47dd-986f-d478006ecb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:25.003565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf6d10-542f-47dd-986f-d478006ecb2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:25.003847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf6d10-542f-47dd-986f-d478006ecb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:25.004158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf6d10-542f-47dd-986f-d478006ecb2e admin admin] PATH_INFO: `/v3/users/d8e701158e66449c846aac6b67331cf6` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:25.005464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf6d10-542f-47dd-986f-d478006ecb2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e701158e66449c846aac6b67331cf6)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:25.007396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf6d10-542f-47dd-986f-d478006ecb2e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:25.174682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-60cf6d10-542f-47dd-986f-d478006ecb2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e701158e66449c846aac6b67331cf6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:25.218148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:25.219627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:25.251409 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-60cf6d10-542f-47dd-986f-d478006ecb2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e701158e66449c846aac6b67331cf6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:25.252050 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-60cf6d10-542f-47dd-986f-d478006ecb2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e701158e66449c846aac6b67331cf6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:25.259661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-60cf6d10-542f-47dd-986f-d478006ecb2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e701158e66449c846aac6b67331cf6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:25.260897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:14:25.261497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:14:25.273323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-60cf6d10-542f-47dd-986f-d478006ecb2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e701158e66449c846aac6b67331cf6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:25.277149 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6810a8a-c598-423b-ab37-6c500ddd1585 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:25.279595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1243/2491] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:14:24 2022] DELETE /identity/v3/users/d8e701158e66449c846aac6b67331cf6 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:14:25.281036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd00a3-44f4-4022-bf2c-bf60f2faf26f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:25.284566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd00a3-44f4-4022-bf2c-bf60f2faf26f service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:25.284566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd00a3-44f4-4022-bf2c-bf60f2faf26f service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:25.284566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd00a3-44f4-4022-bf2c-bf60f2faf26f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:25.284566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-86dd00a3-44f4-4022-bf2c-bf60f2faf26f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:25.355508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd00a3-44f4-4022-bf2c-bf60f2faf26f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:25.356999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd00a3-44f4-4022-bf2c-bf60f2faf26f service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:25.368253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1249/2492] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:14:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:14:25.733307 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf6d10-542f-47dd-986f-d478006ecb2e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:25.734732 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-60cf6d10-542f-47dd-986f-d478006ecb2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:25.803532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf6d10-542f-47dd-986f-d478006ecb2e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:25.808393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad465f-16ea-4cd1-8471-efb13623ecad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:25.809056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad465f-16ea-4cd1-8471-efb13623ecad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:25.809342 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad465f-16ea-4cd1-8471-efb13623ecad admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:25.809622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad465f-16ea-4cd1-8471-efb13623ecad admin admin] PATH_INFO: `/v3/projects/f0cd72652c954d4ebf3aa00259f4445e` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:25.810773 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad465f-16ea-4cd1-8471-efb13623ecad admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0cd72652c954d4ebf3aa00259f4445e)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:25.812696 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad465f-16ea-4cd1-8471-efb13623ecad admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:25.912718 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5fad465f-16ea-4cd1-8471-efb13623ecad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0cd72652c954d4ebf3aa00259f4445e', 'request_id': 'req-5fad465f-16ea-4cd1-8471-efb13623ecad'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:25.918964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5fad465f-16ea-4cd1-8471-efb13623ecad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0cd72652c954d4ebf3aa00259f4445e', 'request_id': 'req-5fad465f-16ea-4cd1-8471-efb13623ecad'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:26.052873 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5fad465f-16ea-4cd1-8471-efb13623ecad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0cd72652c954d4ebf3aa00259f4445e', 'request_id': 'req-5fad465f-16ea-4cd1-8471-efb13623ecad'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:26.064102 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5fad465f-16ea-4cd1-8471-efb13623ecad admin admin] The token cache is being invalidate because project f0cd72652c954d4ebf3aa00259f4445e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:14:26.065338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5fad465f-16ea-4cd1-8471-efb13623ecad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:26.065338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5fad465f-16ea-4cd1-8471-efb13623ecad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:14:26.067533 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1244/2493] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:14:25 2022] DELETE /identity/v3/projects/f0cd72652c954d4ebf3aa00259f4445e => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:14:26.076858 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef21ca9-6b49-462e-828b-aba02f614fd1 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:26.077156 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef21ca9-6b49-462e-828b-aba02f614fd1 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:26.077440 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef21ca9-6b49-462e-828b-aba02f614fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:26.124688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-2ef21ca9-6b49-462e-828b-aba02f614fd1 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:14:26.125888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2ef21ca9-6b49-462e-828b-aba02f614fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:26.183298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1250/2494] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:14:26.191165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568533-0fb5-4981-bef4-ed8c39b941f3 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:26.191455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568533-0fb5-4981-bef4-ed8c39b941f3 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:26.192316 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568533-0fb5-4981-bef4-ed8c39b941f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:26.211700 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-77568533-0fb5-4981-bef4-ed8c39b941f3 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:14:26.212960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-77568533-0fb5-4981-bef4-ed8c39b941f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:26.220889 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1245/2495] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:14:26.225303 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef21ca9-6b49-462e-828b-aba02f614fd1 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:26.235268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef21ca9-6b49-462e-828b-aba02f614fd1 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:26.240545 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9db27a-41ed-4336-902e-839940f0d7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:26.241181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9db27a-41ed-4336-902e-839940f0d7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:26.241466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9db27a-41ed-4336-902e-839940f0d7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:26.241737 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9db27a-41ed-4336-902e-839940f0d7b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:26.242383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9db27a-41ed-4336-902e-839940f0d7b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:14:26.242689 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9db27a-41ed-4336-902e-839940f0d7b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:26.244298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9db27a-41ed-4336-902e-839940f0d7b6 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:26.260702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1251/2496] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:14:26.261823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77568533-0fb5-4981-bef4-ed8c39b941f3 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:26.271590 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77568533-0fb5-4981-bef4-ed8c39b941f3 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:26.276306 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036c29f-6c11-4368-98c6-0615bdddb2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:26.276934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036c29f-6c11-4368-98c6-0615bdddb2a0 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:26.277213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036c29f-6c11-4368-98c6-0615bdddb2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:26.277484 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036c29f-6c11-4368-98c6-0615bdddb2a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:26.278117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036c29f-6c11-4368-98c6-0615bdddb2a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:26.279993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036c29f-6c11-4368-98c6-0615bdddb2a0 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:26.382043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1246/2497] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:14:26 2022] POST /identity/v3/projects => generated 391 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:14:26.386112 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9db27a-41ed-4336-902e-839940f0d7b6 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:26.396001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9db27a-41ed-4336-902e-839940f0d7b6 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:26.438567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:14:26.454164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58b032-7df7-4252-b644-69893e1b147f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:26.454766 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58b032-7df7-4252-b644-69893e1b147f admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:26.455049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58b032-7df7-4252-b644-69893e1b147f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:26.455347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58b032-7df7-4252-b644-69893e1b147f admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:26.455970 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58b032-7df7-4252-b644-69893e1b147f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:26.457923 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58b032-7df7-4252-b644-69893e1b147f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:26.529042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1252/2498] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:14:26 2022] POST /identity/v3/users => generated 296 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:14:26.533005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036c29f-6c11-4368-98c6-0615bdddb2a0 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:26.547829 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036c29f-6c11-4368-98c6-0615bdddb2a0 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:26.549340 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e1d52-8446-4851-9dd4-934e18e17abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:26.550088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e1d52-8446-4851-9dd4-934e18e17abd admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:26.550436 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e1d52-8446-4851-9dd4-934e18e17abd admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:26.550802 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e1d52-8446-4851-9dd4-934e18e17abd admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:26.552287 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e1d52-8446-4851-9dd4-934e18e17abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:26.554092 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e1d52-8446-4851-9dd4-934e18e17abd admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:26.569970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1247/2499] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:14:26.571098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58b032-7df7-4252-b644-69893e1b147f admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:26.582689 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58b032-7df7-4252-b644-69893e1b147f admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:26.588397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5815614-eeb0-46f9-b3e0-7fe1a7a5d729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:26.589035 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5815614-eeb0-46f9-b3e0-7fe1a7a5d729 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:26.589322 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5815614-eeb0-46f9-b3e0-7fe1a7a5d729 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:26.589600 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5815614-eeb0-46f9-b3e0-7fe1a7a5d729 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:26.590253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5815614-eeb0-46f9-b3e0-7fe1a7a5d729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:26.591857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5815614-eeb0-46f9-b3e0-7fe1a7a5d729 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:26.609938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1253/2500] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:14:26.614586 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e1d52-8446-4851-9dd4-934e18e17abd admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:26.624695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e1d52-8446-4851-9dd4-934e18e17abd admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:26.629324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cf9cd-e105-412a-961b-9a92d2db2c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:26.630012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cf9cd-e105-412a-961b-9a92d2db2c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:26.630312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cf9cd-e105-412a-961b-9a92d2db2c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:26.630569 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cf9cd-e105-412a-961b-9a92d2db2c2e admin admin] PATH_INFO: `/v3/projects/4123f99fac0349699118351a05998a96/users/23a87870fdaf48b0b7b55193f41f8eb0/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:26.645358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7cf9cd-e105-412a-961b-9a92d2db2c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4123f99fac0349699118351a05998a96, user_id=23a87870fdaf48b0b7b55193f41f8eb0, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:26.647985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7cf9cd-e105-412a-961b-9a92d2db2c2e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:26.715616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:14:26.721453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1248/2501] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:14:26 2022] PUT /identity/v3/projects/4123f99fac0349699118351a05998a96/users/23a87870fdaf48b0b7b55193f41f8eb0/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:14:26.790150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8b386-3166-49b6-aa4a-dbe322539374 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:26.790441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8b386-3166-49b6-aa4a-dbe322539374 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:26.790718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8b386-3166-49b6-aa4a-dbe322539374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:26.819061 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a5c8b386-3166-49b6-aa4a-dbe322539374 None None] MFA Rules not processed for user `23a87870fdaf48b0b7b55193f41f8eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:14:26.820489 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a5c8b386-3166-49b6-aa4a-dbe322539374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:26.886210 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1254/2502] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:14:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:14:26.897897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cf9cd-e105-412a-961b-9a92d2db2c2e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:14:26.899504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-fd7cf9cd-e105-412a-961b-9a92d2db2c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:14:26.965769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cf9cd-e105-412a-961b-9a92d2db2c2e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:14:26.969382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45f8f3-09f4-41e2-a4db-8e0eea68828d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:14:26.969972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45f8f3-09f4-41e2-a4db-8e0eea68828d service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:14:26.970249 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45f8f3-09f4-41e2-a4db-8e0eea68828d service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:14:26.970519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45f8f3-09f4-41e2-a4db-8e0eea68828d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:14:26.979124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45f8f3-09f4-41e2-a4db-8e0eea68828d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:14:26.980533 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45f8f3-09f4-41e2-a4db-8e0eea68828d service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:14:26.989060 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1249/2503] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:14:26 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:14:36.883212 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=47,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:14:36.985130 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=32 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:15:20.524258 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8b386-3166-49b6-aa4a-dbe322539374 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:15:20.534044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8b386-3166-49b6-aa4a-dbe322539374 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:15:20.538328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6f9ae-9f15-437e-bead-a7340bace391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:15:20.538938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6f9ae-9f15-437e-bead-a7340bace391 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:15:20.539223 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6f9ae-9f15-437e-bead-a7340bace391 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:15:20.539501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6f9ae-9f15-437e-bead-a7340bace391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:15:20.541561 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-67a6f9ae-9f15-437e-bead-a7340bace391 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:15:20.628081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6f9ae-9f15-437e-bead-a7340bace391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:15:20.629599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6f9ae-9f15-437e-bead-a7340bace391 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:15:20.647991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1255/2504] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:15:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:15:27.275340 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45f8f3-09f4-41e2-a4db-8e0eea68828d service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:15:27.276881 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5c45f8f3-09f4-41e2-a4db-8e0eea68828d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:15:27.345127 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45f8f3-09f4-41e2-a4db-8e0eea68828d service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:15:27.350165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643bd92-8a67-469d-b9c1-0ebd1d55b5ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:15:27.350757 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643bd92-8a67-469d-b9c1-0ebd1d55b5ee service placement] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:15:27.351039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643bd92-8a67-469d-b9c1-0ebd1d55b5ee service placement] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:15:27.351315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643bd92-8a67-469d-b9c1-0ebd1d55b5ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:15:27.353743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-a643bd92-8a67-469d-b9c1-0ebd1d55b5ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:15:27.365109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643bd92-8a67-469d-b9c1-0ebd1d55b5ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:15:27.366725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643bd92-8a67-469d-b9c1-0ebd1d55b5ee service placement] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:15:27.377120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1250/2505] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:15:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:15:30.644838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=41 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:15:37.373000 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=32 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:16:21.464765 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6f9ae-9f15-437e-bead-a7340bace391 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:16:21.466473 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-67a6f9ae-9f15-437e-bead-a7340bace391 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:16:21.522982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6f9ae-9f15-437e-bead-a7340bace391 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:16:21.532929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a0634-d203-47b3-99e1-7a257e6e3908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:16:21.534007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a0634-d203-47b3-99e1-7a257e6e3908 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:16:21.534007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a0634-d203-47b3-99e1-7a257e6e3908 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:16:21.534239 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a0634-d203-47b3-99e1-7a257e6e3908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:16:21.535948 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-cd6a0634-d203-47b3-99e1-7a257e6e3908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:16:21.599853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a0634-d203-47b3-99e1-7a257e6e3908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:16:21.601319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a0634-d203-47b3-99e1-7a257e6e3908 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:16:21.610761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1256/2506] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:16:25.498037 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643bd92-8a67-469d-b9c1-0ebd1d55b5ee service placement] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:16:25.500846 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-a643bd92-8a67-469d-b9c1-0ebd1d55b5ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:16:25.513872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643bd92-8a67-469d-b9c1-0ebd1d55b5ee service placement] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:16:25.520955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b174ef9-9795-45df-bb3f-7ed254b17e6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:16:25.521574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b174ef9-9795-45df-bb3f-7ed254b17e6a service placement] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:16:25.521856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b174ef9-9795-45df-bb3f-7ed254b17e6a service placement] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:16:25.522132 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b174ef9-9795-45df-bb3f-7ed254b17e6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:16:25.523911 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2b174ef9-9795-45df-bb3f-7ed254b17e6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:16:25.539058 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b174ef9-9795-45df-bb3f-7ed254b17e6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:16:25.540582 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b174ef9-9795-45df-bb3f-7ed254b17e6a service placement] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:16:25.558252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1251/2507] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:16:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:16:29.334767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a0634-d203-47b3-99e1-7a257e6e3908 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:16:29.341010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-cd6a0634-d203-47b3-99e1-7a257e6e3908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:16:29.350984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a0634-d203-47b3-99e1-7a257e6e3908 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:16:29.357173 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cce89-5a53-4004-8420-c9a77d41af59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:16:29.357896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cce89-5a53-4004-8420-c9a77d41af59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:16:29.358179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cce89-5a53-4004-8420-c9a77d41af59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:16:29.358450 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cce89-5a53-4004-8420-c9a77d41af59 admin admin] PATH_INFO: `/v3/users/9a4f3bfb16c6418cbdc1b6e8d4457a9a` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:16:29.359633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7cce89-5a53-4004-8420-c9a77d41af59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a4f3bfb16c6418cbdc1b6e8d4457a9a)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:16:29.361661 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7cce89-5a53-4004-8420-c9a77d41af59 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:16:29.539369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3a7cce89-5a53-4004-8420-c9a77d41af59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a4f3bfb16c6418cbdc1b6e8d4457a9a'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:29.551991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3a7cce89-5a53-4004-8420-c9a77d41af59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a4f3bfb16c6418cbdc1b6e8d4457a9a'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:29.644887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3a7cce89-5a53-4004-8420-c9a77d41af59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a4f3bfb16c6418cbdc1b6e8d4457a9a'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:29.645505 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3a7cce89-5a53-4004-8420-c9a77d41af59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a4f3bfb16c6418cbdc1b6e8d4457a9a'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:29.659172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3a7cce89-5a53-4004-8420-c9a77d41af59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a4f3bfb16c6418cbdc1b6e8d4457a9a'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:29.665390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1257/2508] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:16:29 2022] DELETE /identity/v3/users/9a4f3bfb16c6418cbdc1b6e8d4457a9a => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:16:29.923511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b174ef9-9795-45df-bb3f-7ed254b17e6a service placement] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:16:29.936891 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b174ef9-9795-45df-bb3f-7ed254b17e6a service placement] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:16:29.996458 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:16:29.997161 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:16:30.003256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:16:30.003256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] PATH_INFO: `/v3/projects/65b31c186fea438d9d17af25036099e4` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:16:30.003256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b31c186fea438d9d17af25036099e4)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:16:30.003256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:16:30.131547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b31c186fea438d9d17af25036099e4', 'request_id': 'req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:30.138434 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b31c186fea438d9d17af25036099e4', 'request_id': 'req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:30.224828 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b31c186fea438d9d17af25036099e4', 'request_id': 'req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:30.236312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] The token cache is being invalidate because project 65b31c186fea438d9d17af25036099e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:16:30.236602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:30.237279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:30.238776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1252/2509] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:16:29 2022] DELETE /identity/v3/projects/65b31c186fea438d9d17af25036099e4 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:16:30.242762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cce89-5a53-4004-8420-c9a77d41af59 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:16:30.244166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3a7cce89-5a53-4004-8420-c9a77d41af59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:16:30.257883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cce89-5a53-4004-8420-c9a77d41af59 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:16:30.275831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:16:30.314731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789b5d8-6caa-405b-ad2a-5febaa562fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:16:30.315359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789b5d8-6caa-405b-ad2a-5febaa562fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:16:30.315645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789b5d8-6caa-405b-ad2a-5febaa562fee admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:16:30.315928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789b5d8-6caa-405b-ad2a-5febaa562fee admin admin] PATH_INFO: `/v3/users/fb2e0176b51b4794ae0397697c08aa96` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:16:30.329379 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789b5d8-6caa-405b-ad2a-5febaa562fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb2e0176b51b4794ae0397697c08aa96)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:16:30.331320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789b5d8-6caa-405b-ad2a-5febaa562fee admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:16:30.534014 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2789b5d8-6caa-405b-ad2a-5febaa562fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb2e0176b51b4794ae0397697c08aa96'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:30.541008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2789b5d8-6caa-405b-ad2a-5febaa562fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb2e0176b51b4794ae0397697c08aa96'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:30.605803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2789b5d8-6caa-405b-ad2a-5febaa562fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb2e0176b51b4794ae0397697c08aa96'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:30.606447 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2789b5d8-6caa-405b-ad2a-5febaa562fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb2e0176b51b4794ae0397697c08aa96'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:30.620781 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2789b5d8-6caa-405b-ad2a-5febaa562fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb2e0176b51b4794ae0397697c08aa96'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:30.625491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1258/2510] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:16:30 2022] DELETE /identity/v3/users/fb2e0176b51b4794ae0397697c08aa96 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:16:30.889666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:16:30.891169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:16:30.906789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0be1c-1ded-42fa-b3c4-6ef85d6999e8 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:16:30.965080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:16:30.965960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:16:30.966254 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:16:30.966531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] PATH_INFO: `/v3/projects/4894bc5a1ec34931b5e4203f2c4b2544` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:16:30.976925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4894bc5a1ec34931b5e4203f2c4b2544)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:16:30.978795 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:16:30.989435 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:16:31.087242 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4894bc5a1ec34931b5e4203f2c4b2544', 'request_id': 'req-b8febcd9-481f-4284-a0a9-88ae8221076c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:31.094676 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4894bc5a1ec34931b5e4203f2c4b2544', 'request_id': 'req-b8febcd9-481f-4284-a0a9-88ae8221076c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:31.178502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4894bc5a1ec34931b5e4203f2c4b2544', 'request_id': 'req-b8febcd9-481f-4284-a0a9-88ae8221076c'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:31.189633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] The token cache is being invalidate because project 4894bc5a1ec34931b5e4203f2c4b2544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:16:31.189922 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:31.190543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:16:31.192083 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1253/2511] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:16:30 2022] DELETE /identity/v3/projects/4894bc5a1ec34931b5e4203f2c4b2544 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:16:40.621961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=48,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:16:41.186891 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:18:22.104911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789b5d8-6caa-405b-ad2a-5febaa562fee admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:22.106333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2789b5d8-6caa-405b-ad2a-5febaa562fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:22.163028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789b5d8-6caa-405b-ad2a-5febaa562fee admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:22.211136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d2ccf-9e19-4a01-a5b5-31a843d621da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:22.211736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d2ccf-9e19-4a01-a5b5-31a843d621da service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:22.212018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d2ccf-9e19-4a01-a5b5-31a843d621da service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:22.212293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d2ccf-9e19-4a01-a5b5-31a843d621da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:22.214056 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-546d2ccf-9e19-4a01-a5b5-31a843d621da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:22.275699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d2ccf-9e19-4a01-a5b5-31a843d621da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:22.277278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d2ccf-9e19-4a01-a5b5-31a843d621da service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:22.287064 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1259/2512] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:18:24.681645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:24.683221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:24.766421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8febcd9-481f-4284-a0a9-88ae8221076c admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:24.774572 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927bdb2-7003-44e0-9c50-cfe32114f947 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:24.775213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927bdb2-7003-44e0-9c50-cfe32114f947 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:24.775502 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927bdb2-7003-44e0-9c50-cfe32114f947 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:24.775777 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927bdb2-7003-44e0-9c50-cfe32114f947 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:24.777758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-1927bdb2-7003-44e0-9c50-cfe32114f947 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:24.794761 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927bdb2-7003-44e0-9c50-cfe32114f947 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:24.796391 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927bdb2-7003-44e0-9c50-cfe32114f947 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:24.809953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1254/2513] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:18:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:18:26.069372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d2ccf-9e19-4a01-a5b5-31a843d621da service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:26.071128 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-546d2ccf-9e19-4a01-a5b5-31a843d621da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:26.084947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d2ccf-9e19-4a01-a5b5-31a843d621da service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:26.089892 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be84af-e0f0-4a1f-a305-7957676ca641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:26.090517 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be84af-e0f0-4a1f-a305-7957676ca641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:26.090804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be84af-e0f0-4a1f-a305-7957676ca641 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:26.091079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be84af-e0f0-4a1f-a305-7957676ca641 admin admin] PATH_INFO: `/v3/users/aee01beeba5b48c3b6c25489743de40e` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:26.092221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be84af-e0f0-4a1f-a305-7957676ca641 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aee01beeba5b48c3b6c25489743de40e)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:26.094353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be84af-e0f0-4a1f-a305-7957676ca641 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:26.286583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f8be84af-e0f0-4a1f-a305-7957676ca641 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aee01beeba5b48c3b6c25489743de40e'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:26.293320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f8be84af-e0f0-4a1f-a305-7957676ca641 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aee01beeba5b48c3b6c25489743de40e'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:26.297090 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:18:26.377638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f8be84af-e0f0-4a1f-a305-7957676ca641 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aee01beeba5b48c3b6c25489743de40e'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:26.378228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f8be84af-e0f0-4a1f-a305-7957676ca641 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aee01beeba5b48c3b6c25489743de40e'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:26.390191 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f8be84af-e0f0-4a1f-a305-7957676ca641 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aee01beeba5b48c3b6c25489743de40e'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:26.395893 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1260/2514] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:18:26 2022] DELETE /identity/v3/users/aee01beeba5b48c3b6c25489743de40e => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:18:26.642029 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927bdb2-7003-44e0-9c50-cfe32114f947 service nova] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:26.653468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927bdb2-7003-44e0-9c50-cfe32114f947 service nova] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:26.720786 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:26.721473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:26.721758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:26.722083 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7 admin admin] PATH_INFO: `/v3/projects/ee7ef4f035fd402fb7e21a21c40e7c49` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:26.723196 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee7ef4f035fd402fb7e21a21c40e7c49)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:26.725208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:26.827928 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee7ef4f035fd402fb7e21a21c40e7c49', 'request_id': 'req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:26.834106 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee7ef4f035fd402fb7e21a21c40e7c49', 'request_id': 'req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:26.906573 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee7ef4f035fd402fb7e21a21c40e7c49', 'request_id': 'req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:26.917106 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7 admin admin] The token cache is being invalidate because project ee7ef4f035fd402fb7e21a21c40e7c49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:18:26.920723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:26.920723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-91c9ecd9-60ee-40eb-a19b-c297d8dc6ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:26.920723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1255/2515] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:18:26 2022] DELETE /identity/v3/projects/ee7ef4f035fd402fb7e21a21c40e7c49 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:18:26.925552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9b4cb-19fb-4e13-b38a-33a5c47a32f6 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:26.925870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9b4cb-19fb-4e13-b38a-33a5c47a32f6 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:26.926147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9b4cb-19fb-4e13-b38a-33a5c47a32f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:26.946253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-6bf9b4cb-19fb-4e13-b38a-33a5c47a32f6 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:18:26.947476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6bf9b4cb-19fb-4e13-b38a-33a5c47a32f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:27.005929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1261/2516] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:18:27.013705 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172e0f3-789c-4b48-ad20-bfcf376bc0b7 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:27.013992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172e0f3-789c-4b48-ad20-bfcf376bc0b7 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:27.014268 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172e0f3-789c-4b48-ad20-bfcf376bc0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:27.035436 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-2172e0f3-789c-4b48-ad20-bfcf376bc0b7 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:18:27.036627 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2172e0f3-789c-4b48-ad20-bfcf376bc0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:27.044571 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1256/2517] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:18:27.049440 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9b4cb-19fb-4e13-b38a-33a5c47a32f6 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:27.059216 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9b4cb-19fb-4e13-b38a-33a5c47a32f6 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:27.064366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56799288-e3f3-4b51-8173-06cf7b76dbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:27.064997 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56799288-e3f3-4b51-8173-06cf7b76dbff admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:27.065278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56799288-e3f3-4b51-8173-06cf7b76dbff admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:27.065550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56799288-e3f3-4b51-8173-06cf7b76dbff admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:27.066188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56799288-e3f3-4b51-8173-06cf7b76dbff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:18:27.066493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56799288-e3f3-4b51-8173-06cf7b76dbff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:27.068093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56799288-e3f3-4b51-8173-06cf7b76dbff admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:27.079511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1262/2518] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:18:27.083625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172e0f3-789c-4b48-ad20-bfcf376bc0b7 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:27.092526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172e0f3-789c-4b48-ad20-bfcf376bc0b7 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:27.097416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d78669-c26e-42f4-b80f-533a902b15d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:27.098001 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d78669-c26e-42f4-b80f-533a902b15d7 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:27.098279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d78669-c26e-42f4-b80f-533a902b15d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:27.098550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d78669-c26e-42f4-b80f-533a902b15d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:27.099165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d78669-c26e-42f4-b80f-533a902b15d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:27.101077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d78669-c26e-42f4-b80f-533a902b15d7 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:27.106411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:18:27.213825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1257/2519] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:18:27 2022] POST /identity/v3/projects => generated 363 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:18:27.218154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56799288-e3f3-4b51-8173-06cf7b76dbff admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:27.228076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56799288-e3f3-4b51-8173-06cf7b76dbff admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:27.287501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658f7c6-ef84-437d-b263-ae263c5e2ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:27.287672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658f7c6-ef84-437d-b263-ae263c5e2ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:27.287875 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658f7c6-ef84-437d-b263-ae263c5e2ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:27.288148 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658f7c6-ef84-437d-b263-ae263c5e2ba3 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:27.288799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658f7c6-ef84-437d-b263-ae263c5e2ba3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:27.290750 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658f7c6-ef84-437d-b263-ae263c5e2ba3 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:27.378636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1263/2520] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:18:27 2022] POST /identity/v3/users => generated 282 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:18:27.382664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d78669-c26e-42f4-b80f-533a902b15d7 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:27.393652 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d78669-c26e-42f4-b80f-533a902b15d7 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:27.398362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895cf13-ba8f-4b18-ae4e-471ff2b95799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:27.398958 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895cf13-ba8f-4b18-ae4e-471ff2b95799 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:27.399239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895cf13-ba8f-4b18-ae4e-471ff2b95799 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:27.399513 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895cf13-ba8f-4b18-ae4e-471ff2b95799 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:27.400153 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895cf13-ba8f-4b18-ae4e-471ff2b95799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:27.401769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895cf13-ba8f-4b18-ae4e-471ff2b95799 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:27.411686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1258/2521] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:18:27.415644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658f7c6-ef84-437d-b263-ae263c5e2ba3 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:27.425584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658f7c6-ef84-437d-b263-ae263c5e2ba3 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:27.430401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7506024-b0c2-4fe6-a6d1-68c42d66ea6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:27.430986 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7506024-b0c2-4fe6-a6d1-68c42d66ea6f admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:27.431270 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7506024-b0c2-4fe6-a6d1-68c42d66ea6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:27.431543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7506024-b0c2-4fe6-a6d1-68c42d66ea6f admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:27.432211 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7506024-b0c2-4fe6-a6d1-68c42d66ea6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:27.433829 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7506024-b0c2-4fe6-a6d1-68c42d66ea6f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:27.445172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1264/2522] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:18:27.449169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895cf13-ba8f-4b18-ae4e-471ff2b95799 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:27.459788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895cf13-ba8f-4b18-ae4e-471ff2b95799 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:27.464455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f42c9-8c40-47b6-89a7-44f53ffa9c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:27.465070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f42c9-8c40-47b6-89a7-44f53ffa9c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:27.465350 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f42c9-8c40-47b6-89a7-44f53ffa9c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:27.465623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f42c9-8c40-47b6-89a7-44f53ffa9c42 admin admin] PATH_INFO: `/v3/projects/96f760a84fa74e738456811c583f06f2/users/443432b323b242bdb5a178eddad6ab64/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:27.480131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f42c9-8c40-47b6-89a7-44f53ffa9c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f760a84fa74e738456811c583f06f2, user_id=443432b323b242bdb5a178eddad6ab64, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:27.482800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f42c9-8c40-47b6-89a7-44f53ffa9c42 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:27.548816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1259/2523] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:18:27 2022] PUT /identity/v3/projects/96f760a84fa74e738456811c583f06f2/users/443432b323b242bdb5a178eddad6ab64/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:18:27.560962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675a5eb-0093-43f3-ab73-f3bb3c0247e2 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:27.561253 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675a5eb-0093-43f3-ab73-f3bb3c0247e2 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:27.561523 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675a5eb-0093-43f3-ab73-f3bb3c0247e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:27.589654 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9675a5eb-0093-43f3-ab73-f3bb3c0247e2 None None] MFA Rules not processed for user `443432b323b242bdb5a178eddad6ab64`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:18:27.590872 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9675a5eb-0093-43f3-ab73-f3bb3c0247e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:27.622382 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:18:27.654244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1265/2524] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:18:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:18:27.659065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f42c9-8c40-47b6-89a7-44f53ffa9c42 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:27.720993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f42c9-8c40-47b6-89a7-44f53ffa9c42 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:27.725639 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaca7da-5685-46c4-8dd9-bb2dee0c6cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:27.726242 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaca7da-5685-46c4-8dd9-bb2dee0c6cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:27.726527 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaca7da-5685-46c4-8dd9-bb2dee0c6cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:27.726820 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaca7da-5685-46c4-8dd9-bb2dee0c6cb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:27.727438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaca7da-5685-46c4-8dd9-bb2dee0c6cb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:27.729401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaca7da-5685-46c4-8dd9-bb2dee0c6cb9 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:27.841832 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1260/2525] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:18:27 2022] POST /identity/v3/projects => generated 363 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:18:27.846629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675a5eb-0093-43f3-ab73-f3bb3c0247e2 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:27.857461 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675a5eb-0093-43f3-ab73-f3bb3c0247e2 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:27.913113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42778f7f-60b8-49fe-b96c-346101a551ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:27.913719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42778f7f-60b8-49fe-b96c-346101a551ce admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:27.914005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42778f7f-60b8-49fe-b96c-346101a551ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:27.914283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42778f7f-60b8-49fe-b96c-346101a551ce admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:27.914913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42778f7f-60b8-49fe-b96c-346101a551ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:27.916853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42778f7f-60b8-49fe-b96c-346101a551ce admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:28.018071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1266/2526] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:18:27 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:18:28.021976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaca7da-5685-46c4-8dd9-bb2dee0c6cb9 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:28.033060 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaca7da-5685-46c4-8dd9-bb2dee0c6cb9 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:28.037639 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a055137-b4b6-40fd-b624-de324b234e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:28.038286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a055137-b4b6-40fd-b624-de324b234e0b admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:28.038572 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a055137-b4b6-40fd-b624-de324b234e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:28.038842 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a055137-b4b6-40fd-b624-de324b234e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:28.039473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a055137-b4b6-40fd-b624-de324b234e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:28.041097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a055137-b4b6-40fd-b624-de324b234e0b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:28.051914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1261/2527] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:18:28.056410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42778f7f-60b8-49fe-b96c-346101a551ce admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:28.066946 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42778f7f-60b8-49fe-b96c-346101a551ce admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:28.071568 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5570a-9c84-4bba-9439-d0f183bfa6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:28.072348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5570a-9c84-4bba-9439-d0f183bfa6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:28.072631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5570a-9c84-4bba-9439-d0f183bfa6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:28.072929 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5570a-9c84-4bba-9439-d0f183bfa6d6 admin admin] PATH_INFO: `/v3/projects/a3caed5b20884c8a903bdcc584f55c87/users/e2193689242f47d6a99b1b65c895c705/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:28.088027 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5570a-9c84-4bba-9439-d0f183bfa6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3caed5b20884c8a903bdcc584f55c87, user_id=e2193689242f47d6a99b1b65c895c705, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:28.090752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5570a-9c84-4bba-9439-d0f183bfa6d6 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:28.165619 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1267/2528] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:18:28 2022] PUT /identity/v3/projects/a3caed5b20884c8a903bdcc584f55c87/users/e2193689242f47d6a99b1b65c895c705/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:18:28.169558 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a055137-b4b6-40fd-b624-de324b234e0b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:28.179784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a055137-b4b6-40fd-b624-de324b234e0b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:28.238265 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211655e0-9772-4ec4-81f9-735116a2ad26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:28.238898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211655e0-9772-4ec4-81f9-735116a2ad26 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:28.239185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211655e0-9772-4ec4-81f9-735116a2ad26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:28.239462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211655e0-9772-4ec4-81f9-735116a2ad26 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:28.240109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211655e0-9772-4ec4-81f9-735116a2ad26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:28.241764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211655e0-9772-4ec4-81f9-735116a2ad26 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:28.252267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1262/2529] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:18:28.256427 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5570a-9c84-4bba-9439-d0f183bfa6d6 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:28.273046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5570a-9c84-4bba-9439-d0f183bfa6d6 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:28.273315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b4348-6ed9-4d30-bd77-f3522d1518a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:28.273974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b4348-6ed9-4d30-bd77-f3522d1518a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:28.274257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b4348-6ed9-4d30-bd77-f3522d1518a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:28.274533 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b4348-6ed9-4d30-bd77-f3522d1518a0 admin admin] PATH_INFO: `/v3/projects/a3caed5b20884c8a903bdcc584f55c87/users/e2193689242f47d6a99b1b65c895c705/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:28.276450 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b4348-6ed9-4d30-bd77-f3522d1518a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3caed5b20884c8a903bdcc584f55c87, user_id=e2193689242f47d6a99b1b65c895c705, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:28.279218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b4348-6ed9-4d30-bd77-f3522d1518a0 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:28.289377 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1268/2530] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:18:28 2022] PUT /identity/v3/projects/a3caed5b20884c8a903bdcc584f55c87/users/e2193689242f47d6a99b1b65c895c705/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:18:28.299788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7c7bb-6af5-4564-add4-3c04b669ff5a None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:28.300078 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7c7bb-6af5-4564-add4-3c04b669ff5a None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:28.300378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7c7bb-6af5-4564-add4-3c04b669ff5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:28.328281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-bfa7c7bb-6af5-4564-add4-3c04b669ff5a None None] MFA Rules not processed for user `e2193689242f47d6a99b1b65c895c705`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:18:28.330304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-bfa7c7bb-6af5-4564-add4-3c04b669ff5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:28.340059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:18:28.405106 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1263/2531] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:18:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:18:28.437558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b4348-6ed9-4d30-bd77-f3522d1518a0 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:28.439953 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-de5b4348-6ed9-4d30-bd77-f3522d1518a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:28.496732 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b4348-6ed9-4d30-bd77-f3522d1518a0 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:28.501378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbc4a9-4c50-4060-9a74-2593e44676b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:28.502141 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbc4a9-4c50-4060-9a74-2593e44676b4 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:28.502426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbc4a9-4c50-4060-9a74-2593e44676b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:28.502704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbc4a9-4c50-4060-9a74-2593e44676b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:28.511542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbc4a9-4c50-4060-9a74-2593e44676b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:28.513003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbc4a9-4c50-4060-9a74-2593e44676b4 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:28.522518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1269/2532] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:18:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:18:29.520994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7c7bb-6af5-4564-add4-3c04b669ff5a None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:29.591133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7c7bb-6af5-4564-add4-3c04b669ff5a None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:29.596985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe35141-9c4b-4774-a5cb-14426f4836bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:29.597699 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe35141-9c4b-4774-a5cb-14426f4836bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:29.598050 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe35141-9c4b-4774-a5cb-14426f4836bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:29.598395 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe35141-9c4b-4774-a5cb-14426f4836bd admin admin] PATH_INFO: `/v3/users/443432b323b242bdb5a178eddad6ab64` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:29.599583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe35141-9c4b-4774-a5cb-14426f4836bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=443432b323b242bdb5a178eddad6ab64)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:29.601700 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe35141-9c4b-4774-a5cb-14426f4836bd admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:29.802512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ffe35141-9c4b-4774-a5cb-14426f4836bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '443432b323b242bdb5a178eddad6ab64'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:29.878237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ffe35141-9c4b-4774-a5cb-14426f4836bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '443432b323b242bdb5a178eddad6ab64'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:29.878851 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ffe35141-9c4b-4774-a5cb-14426f4836bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '443432b323b242bdb5a178eddad6ab64'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:29.886097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ffe35141-9c4b-4774-a5cb-14426f4836bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '443432b323b242bdb5a178eddad6ab64'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:29.898547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ffe35141-9c4b-4774-a5cb-14426f4836bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '443432b323b242bdb5a178eddad6ab64'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:29.904408 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1264/2533] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:18:29 2022] DELETE /identity/v3/users/443432b323b242bdb5a178eddad6ab64 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:18:29.909225 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a828f30-56c6-4c48-a340-e2f74bd746b2 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:29.909734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a828f30-56c6-4c48-a340-e2f74bd746b2 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:29.909857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a828f30-56c6-4c48-a340-e2f74bd746b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:29.928310 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-8a828f30-56c6-4c48-a340-e2f74bd746b2 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:18:29.929531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8a828f30-56c6-4c48-a340-e2f74bd746b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:29.954559 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:18:29.995936 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1270/2534] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:18:30.007004 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe35141-9c4b-4774-a5cb-14426f4836bd admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:30.008759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ffe35141-9c4b-4774-a5cb-14426f4836bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:30.045095 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:18:30.074540 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe35141-9c4b-4774-a5cb-14426f4836bd admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:30.078186 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83639096-0db6-4b3c-aade-04a7131ee670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:30.078878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83639096-0db6-4b3c-aade-04a7131ee670 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:30.079157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83639096-0db6-4b3c-aade-04a7131ee670 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:30.079435 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83639096-0db6-4b3c-aade-04a7131ee670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:30.088427 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83639096-0db6-4b3c-aade-04a7131ee670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:30.089932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83639096-0db6-4b3c-aade-04a7131ee670 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:30.099553 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1265/2535] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:18:30.487174 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a828f30-56c6-4c48-a340-e2f74bd746b2 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:30.488723 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8a828f30-56c6-4c48-a340-e2f74bd746b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:30.502180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a828f30-56c6-4c48-a340-e2f74bd746b2 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:30.509400 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:30.510208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:30.511351 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:30.512059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] PATH_INFO: `/v3/projects/96f760a84fa74e738456811c583f06f2` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:30.513988 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96f760a84fa74e738456811c583f06f2)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:30.519102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:30.601725 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96f760a84fa74e738456811c583f06f2', 'request_id': 'req-e52cf5fc-38ba-46d0-be8c-d46355a81b30'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:30.608026 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96f760a84fa74e738456811c583f06f2', 'request_id': 'req-e52cf5fc-38ba-46d0-be8c-d46355a81b30'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:30.614027 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96f760a84fa74e738456811c583f06f2', 'request_id': 'req-e52cf5fc-38ba-46d0-be8c-d46355a81b30'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:30.668494 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] The token cache is being invalidate because project 96f760a84fa74e738456811c583f06f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:18:30.673913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:30.673913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:30.673913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1271/2536] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:18:30 2022] DELETE /identity/v3/projects/96f760a84fa74e738456811c583f06f2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:18:30.676615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83639096-0db6-4b3c-aade-04a7131ee670 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:30.686503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83639096-0db6-4b3c-aade-04a7131ee670 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:30.743347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4cccc2-ab5c-4dd7-b2b7-3dfdd7c42422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:30.743958 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4cccc2-ab5c-4dd7-b2b7-3dfdd7c42422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:30.744237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4cccc2-ab5c-4dd7-b2b7-3dfdd7c42422 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:30.744510 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4cccc2-ab5c-4dd7-b2b7-3dfdd7c42422 admin admin] PATH_INFO: `/v3/users/e2193689242f47d6a99b1b65c895c705` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:30.745637 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4cccc2-ab5c-4dd7-b2b7-3dfdd7c42422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2193689242f47d6a99b1b65c895c705)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:30.747557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4cccc2-ab5c-4dd7-b2b7-3dfdd7c42422 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:30.904688 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5d4cccc2-ab5c-4dd7-b2b7-3dfdd7c42422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2193689242f47d6a99b1b65c895c705'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:30.988630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5d4cccc2-ab5c-4dd7-b2b7-3dfdd7c42422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2193689242f47d6a99b1b65c895c705'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:30.989280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5d4cccc2-ab5c-4dd7-b2b7-3dfdd7c42422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2193689242f47d6a99b1b65c895c705'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:30.996979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5d4cccc2-ab5c-4dd7-b2b7-3dfdd7c42422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2193689242f47d6a99b1b65c895c705'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:31.009871 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5d4cccc2-ab5c-4dd7-b2b7-3dfdd7c42422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2193689242f47d6a99b1b65c895c705'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:31.015714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1266/2537] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:18:30 2022] DELETE /identity/v3/users/e2193689242f47d6a99b1b65c895c705 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:18:31.474361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:31.478931 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:31.489480 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cf5fc-38ba-46d0-be8c-d46355a81b30 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:31.546932 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04815922-1ccd-47b1-bbb4-d09230974d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:31.548149 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04815922-1ccd-47b1-bbb4-d09230974d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:31.548438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04815922-1ccd-47b1-bbb4-d09230974d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:31.548749 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04815922-1ccd-47b1-bbb4-d09230974d84 admin admin] PATH_INFO: `/v3/projects/a3caed5b20884c8a903bdcc584f55c87` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:31.549874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04815922-1ccd-47b1-bbb4-d09230974d84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3caed5b20884c8a903bdcc584f55c87)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:31.551797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04815922-1ccd-47b1-bbb4-d09230974d84 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:31.674563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-04815922-1ccd-47b1-bbb4-d09230974d84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3caed5b20884c8a903bdcc584f55c87', 'request_id': 'req-04815922-1ccd-47b1-bbb4-d09230974d84'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:31.681456 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-04815922-1ccd-47b1-bbb4-d09230974d84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3caed5b20884c8a903bdcc584f55c87', 'request_id': 'req-04815922-1ccd-47b1-bbb4-d09230974d84'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:31.683063 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:18:31.688422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-04815922-1ccd-47b1-bbb4-d09230974d84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3caed5b20884c8a903bdcc584f55c87', 'request_id': 'req-04815922-1ccd-47b1-bbb4-d09230974d84'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:31.862856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-04815922-1ccd-47b1-bbb4-d09230974d84 admin admin] The token cache is being invalidate because project a3caed5b20884c8a903bdcc584f55c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:18:31.863147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-04815922-1ccd-47b1-bbb4-d09230974d84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:31.863767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-04815922-1ccd-47b1-bbb4-d09230974d84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:18:31.865273 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1272/2538] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:18:31 2022] DELETE /identity/v3/projects/a3caed5b20884c8a903bdcc584f55c87 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:18:31.870217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff30d9-137a-40d3-9043-90e24bf79911 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:31.870741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff30d9-137a-40d3-9043-90e24bf79911 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:31.870860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff30d9-137a-40d3-9043-90e24bf79911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:31.892438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-ceff30d9-137a-40d3-9043-90e24bf79911 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:18:31.893656 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ceff30d9-137a-40d3-9043-90e24bf79911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:31.937057 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:18:31.952555 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1267/2539] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:18:31.961793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49137b7-43bd-4171-a07b-961834581236 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:31.962086 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49137b7-43bd-4171-a07b-961834581236 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:31.962365 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49137b7-43bd-4171-a07b-961834581236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:31.982709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-c49137b7-43bd-4171-a07b-961834581236 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:18:31.983927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c49137b7-43bd-4171-a07b-961834581236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:31.992658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1273/2540] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:18:31.996965 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff30d9-137a-40d3-9043-90e24bf79911 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:32.007346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff30d9-137a-40d3-9043-90e24bf79911 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:32.012217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bcd97-d737-44f3-a951-bae75e7348bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:32.012870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bcd97-d737-44f3-a951-bae75e7348bb admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:32.013150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bcd97-d737-44f3-a951-bae75e7348bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:32.013422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bcd97-d737-44f3-a951-bae75e7348bb admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:32.014165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bcd97-d737-44f3-a951-bae75e7348bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:18:32.014470 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bcd97-d737-44f3-a951-bae75e7348bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:32.016115 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bcd97-d737-44f3-a951-bae75e7348bb admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:32.027804 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1268/2541] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:18:32.034853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49137b7-43bd-4171-a07b-961834581236 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:32.044379 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49137b7-43bd-4171-a07b-961834581236 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:32.049767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74763f-271f-47ce-a186-4e8cf41acb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:32.050359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74763f-271f-47ce-a186-4e8cf41acb71 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:32.050643 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74763f-271f-47ce-a186-4e8cf41acb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:32.050920 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74763f-271f-47ce-a186-4e8cf41acb71 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:32.051558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74763f-271f-47ce-a186-4e8cf41acb71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:32.053764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74763f-271f-47ce-a186-4e8cf41acb71 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:32.148245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1274/2542] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:18:32 2022] POST /identity/v3/projects => generated 355 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:18:32.152305 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bcd97-d737-44f3-a951-bae75e7348bb admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:32.163007 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bcd97-d737-44f3-a951-bae75e7348bb admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:32.223280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d09b8a-319b-4996-8bd4-b9ff408801c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:32.223901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d09b8a-319b-4996-8bd4-b9ff408801c7 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:32.224171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d09b8a-319b-4996-8bd4-b9ff408801c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:32.224444 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d09b8a-319b-4996-8bd4-b9ff408801c7 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:32.225101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d09b8a-319b-4996-8bd4-b9ff408801c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:32.227028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d09b8a-319b-4996-8bd4-b9ff408801c7 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:32.338984 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1269/2543] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:18:32 2022] POST /identity/v3/users => generated 278 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:18:32.345527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74763f-271f-47ce-a186-4e8cf41acb71 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:32.355886 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74763f-271f-47ce-a186-4e8cf41acb71 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:32.360872 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68580b3d-e10f-4b8d-8fa9-0c17a81e0bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:32.361477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68580b3d-e10f-4b8d-8fa9-0c17a81e0bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:32.361774 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68580b3d-e10f-4b8d-8fa9-0c17a81e0bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:32.362046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68580b3d-e10f-4b8d-8fa9-0c17a81e0bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:32.362702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68580b3d-e10f-4b8d-8fa9-0c17a81e0bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:32.364282 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68580b3d-e10f-4b8d-8fa9-0c17a81e0bd2 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:32.374213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1275/2544] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:18:32.378417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d09b8a-319b-4996-8bd4-b9ff408801c7 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:32.388398 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d09b8a-319b-4996-8bd4-b9ff408801c7 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:32.394047 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aaaeb9-8933-4682-92b5-b25d0eacd93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:32.404107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaaeb9-8933-4682-92b5-b25d0eacd93f admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:32.404107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaaeb9-8933-4682-92b5-b25d0eacd93f admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:32.404107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaaeb9-8933-4682-92b5-b25d0eacd93f admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:32.404107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aaaeb9-8933-4682-92b5-b25d0eacd93f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:32.404107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aaaeb9-8933-4682-92b5-b25d0eacd93f admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:32.409235 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1270/2545] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:18:32.413336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68580b3d-e10f-4b8d-8fa9-0c17a81e0bd2 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:32.423291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68580b3d-e10f-4b8d-8fa9-0c17a81e0bd2 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:32.427857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6060d1-1522-47c1-b512-7325cf01d8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:32.428789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6060d1-1522-47c1-b512-7325cf01d8b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:32.432860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6060d1-1522-47c1-b512-7325cf01d8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:32.432860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6060d1-1522-47c1-b512-7325cf01d8b5 admin admin] PATH_INFO: `/v3/projects/f5f23b4842fa4544bf28efeaa21c9ed3/users/24273cc04c614d13ae47048a0f0ff5b1/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:32.444179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6060d1-1522-47c1-b512-7325cf01d8b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f23b4842fa4544bf28efeaa21c9ed3, user_id=24273cc04c614d13ae47048a0f0ff5b1, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:32.446828 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6060d1-1522-47c1-b512-7325cf01d8b5 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:32.538334 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1276/2546] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:18:32 2022] PUT /identity/v3/projects/f5f23b4842fa4544bf28efeaa21c9ed3/users/24273cc04c614d13ae47048a0f0ff5b1/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:18:32.549215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647bdd6-8956-4f4d-aced-477eb406f138 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:32.549508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647bdd6-8956-4f4d-aced-477eb406f138 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:32.549775 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647bdd6-8956-4f4d-aced-477eb406f138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:32.577385 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-1647bdd6-8956-4f4d-aced-477eb406f138 None None] MFA Rules not processed for user `24273cc04c614d13ae47048a0f0ff5b1`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:18:32.579105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-1647bdd6-8956-4f4d-aced-477eb406f138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:32.645374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1271/2547] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:18:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:18:32.649894 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6060d1-1522-47c1-b512-7325cf01d8b5 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:32.712867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6060d1-1522-47c1-b512-7325cf01d8b5 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:32.717686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e677264-90bd-460e-a73b-7eff7132d662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:32.718287 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e677264-90bd-460e-a73b-7eff7132d662 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:32.718565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e677264-90bd-460e-a73b-7eff7132d662 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:32.718835 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e677264-90bd-460e-a73b-7eff7132d662 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:32.719793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e677264-90bd-460e-a73b-7eff7132d662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:32.721716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e677264-90bd-460e-a73b-7eff7132d662 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:32.807662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1277/2548] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:18:32 2022] POST /identity/v3/projects => generated 355 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:18:32.811511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647bdd6-8956-4f4d-aced-477eb406f138 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:32.822993 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647bdd6-8956-4f4d-aced-477eb406f138 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:32.864608 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:18:32.879744 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6e269-0b72-4f31-925c-6ad74980188b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:32.880380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6e269-0b72-4f31-925c-6ad74980188b admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:32.880613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6e269-0b72-4f31-925c-6ad74980188b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:32.880914 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6e269-0b72-4f31-925c-6ad74980188b admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:32.881588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6e269-0b72-4f31-925c-6ad74980188b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:32.883495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6e269-0b72-4f31-925c-6ad74980188b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:32.948416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1272/2549] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:18:32 2022] POST /identity/v3/users => generated 284 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:18:32.952322 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e677264-90bd-460e-a73b-7eff7132d662 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:32.963659 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e677264-90bd-460e-a73b-7eff7132d662 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:32.968293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e3f90-9f55-4d95-b377-7c654cf23173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:32.968951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e3f90-9f55-4d95-b377-7c654cf23173 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:32.969418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e3f90-9f55-4d95-b377-7c654cf23173 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:32.969893 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e3f90-9f55-4d95-b377-7c654cf23173 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:32.970669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e3f90-9f55-4d95-b377-7c654cf23173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:32.972263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e3f90-9f55-4d95-b377-7c654cf23173 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:32.982656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1278/2550] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:18:32.987629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6e269-0b72-4f31-925c-6ad74980188b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:32.997482 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6e269-0b72-4f31-925c-6ad74980188b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:33.003027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f45abc-4adf-49c4-a268-ee549f570e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:33.003672 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f45abc-4adf-49c4-a268-ee549f570e3a admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:33.003970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f45abc-4adf-49c4-a268-ee549f570e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:33.004248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f45abc-4adf-49c4-a268-ee549f570e3a admin admin] PATH_INFO: `/v3/projects/79816b997700419e84c65b766bfa343a/users/c2b3ff7e29024529913f4a894ac655e8/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:33.018671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f45abc-4adf-49c4-a268-ee549f570e3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=79816b997700419e84c65b766bfa343a, user_id=c2b3ff7e29024529913f4a894ac655e8, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:33.021379 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f45abc-4adf-49c4-a268-ee549f570e3a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:33.068751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1273/2551] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:18:32 2022] PUT /identity/v3/projects/79816b997700419e84c65b766bfa343a/users/c2b3ff7e29024529913f4a894ac655e8/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:18:33.072673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e3f90-9f55-4d95-b377-7c654cf23173 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:33.083047 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e3f90-9f55-4d95-b377-7c654cf23173 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:33.133105 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:18:33.143359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474434e3-97ab-4f90-a096-fa8c5005077d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:33.146983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474434e3-97ab-4f90-a096-fa8c5005077d admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:33.146983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474434e3-97ab-4f90-a096-fa8c5005077d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:33.146983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474434e3-97ab-4f90-a096-fa8c5005077d admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:33.146983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474434e3-97ab-4f90-a096-fa8c5005077d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:33.146983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474434e3-97ab-4f90-a096-fa8c5005077d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:33.158318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1279/2552] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:18:33.162501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f45abc-4adf-49c4-a268-ee549f570e3a admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:33.173389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f45abc-4adf-49c4-a268-ee549f570e3a admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:33.178859 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93692ba2-3c3e-4b16-afcf-9ef07d600151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:33.179623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93692ba2-3c3e-4b16-afcf-9ef07d600151 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:33.179974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93692ba2-3c3e-4b16-afcf-9ef07d600151 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:33.180319 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93692ba2-3c3e-4b16-afcf-9ef07d600151 admin admin] PATH_INFO: `/v3/projects/79816b997700419e84c65b766bfa343a/users/c2b3ff7e29024529913f4a894ac655e8/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:33.182448 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93692ba2-3c3e-4b16-afcf-9ef07d600151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79816b997700419e84c65b766bfa343a, user_id=c2b3ff7e29024529913f4a894ac655e8, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:33.185237 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93692ba2-3c3e-4b16-afcf-9ef07d600151 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:33.208770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1274/2553] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:18:33 2022] PUT /identity/v3/projects/79816b997700419e84c65b766bfa343a/users/c2b3ff7e29024529913f4a894ac655e8/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:18:33.209251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ad7f9-da9f-46d3-a57f-a6ec5c491bca None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:33.209634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ad7f9-da9f-46d3-a57f-a6ec5c491bca None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:33.209916 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ad7f9-da9f-46d3-a57f-a6ec5c491bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:33.240306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-866ad7f9-da9f-46d3-a57f-a6ec5c491bca None None] MFA Rules not processed for user `c2b3ff7e29024529913f4a894ac655e8`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:18:33.241560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-866ad7f9-da9f-46d3-a57f-a6ec5c491bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:33.321251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1280/2554] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:18:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:18:33.328776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93692ba2-3c3e-4b16-afcf-9ef07d600151 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:33.330759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-93692ba2-3c3e-4b16-afcf-9ef07d600151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:18:33.408015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93692ba2-3c3e-4b16-afcf-9ef07d600151 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:33.411556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb9974-0796-4128-929a-f22814e780e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:33.412155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb9974-0796-4128-929a-f22814e780e2 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:33.412432 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb9974-0796-4128-929a-f22814e780e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:33.412742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb9974-0796-4128-929a-f22814e780e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:33.421403 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb9974-0796-4128-929a-f22814e780e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:33.422860 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb9974-0796-4128-929a-f22814e780e2 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:33.432165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1275/2555] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:18:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:18:33.654240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ad7f9-da9f-46d3-a57f-a6ec5c491bca None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:18:33.663853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ad7f9-da9f-46d3-a57f-a6ec5c491bca None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:18:33.667420 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d7a96-2e7d-4caf-bda4-61f4a3fa93bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:18:33.668011 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d7a96-2e7d-4caf-bda4-61f4a3fa93bd service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:18:33.668309 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d7a96-2e7d-4caf-bda4-61f4a3fa93bd service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:18:33.668566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d7a96-2e7d-4caf-bda4-61f4a3fa93bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:18:33.725495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d7a96-2e7d-4caf-bda4-61f4a3fa93bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:18:33.726935 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d7a96-2e7d-4caf-bda4-61f4a3fa93bd service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:18:33.736202 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1281/2556] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:18:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:18:43.428120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=50,keystone:INSERT=5 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:18:43.731687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=67 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:19:22.704243 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb9974-0796-4128-929a-f22814e780e2 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:19:22.713510 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb9974-0796-4128-929a-f22814e780e2 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:19:22.717148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b64d1c-b686-4423-aefc-b6ba01448de4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:19:22.717743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b64d1c-b686-4423-aefc-b6ba01448de4 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:19:22.718022 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b64d1c-b686-4423-aefc-b6ba01448de4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:19:22.718296 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b64d1c-b686-4423-aefc-b6ba01448de4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:19:22.719990 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-27b64d1c-b686-4423-aefc-b6ba01448de4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:19:22.780952 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b64d1c-b686-4423-aefc-b6ba01448de4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:19:22.782344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b64d1c-b686-4423-aefc-b6ba01448de4 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:19:22.791489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1276/2557] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:19:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:19:22.959445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d7a96-2e7d-4caf-bda4-61f4a3fa93bd service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:19:22.961005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-246d7a96-2e7d-4caf-bda4-61f4a3fa93bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:19:23.033444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d7a96-2e7d-4caf-bda4-61f4a3fa93bd service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:19:23.038907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f344c2-1031-4952-9d55-a1c31b924910 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:19:23.039521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f344c2-1031-4952-9d55-a1c31b924910 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:19:23.039800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f344c2-1031-4952-9d55-a1c31b924910 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:19:23.040075 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f344c2-1031-4952-9d55-a1c31b924910 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:19:23.041821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-83f344c2-1031-4952-9d55-a1c31b924910 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:19:23.120093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f344c2-1031-4952-9d55-a1c31b924910 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:19:23.121709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f344c2-1031-4952-9d55-a1c31b924910 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:19:23.139835 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1282/2558] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:19:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:19:24.289215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b64d1c-b686-4423-aefc-b6ba01448de4 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:19:24.290695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-27b64d1c-b686-4423-aefc-b6ba01448de4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:19:24.347070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b64d1c-b686-4423-aefc-b6ba01448de4 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:19:24.350789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1555c-322e-4368-836d-162700f5c6df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:19:24.351394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1555c-322e-4368-836d-162700f5c6df service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:19:24.351674 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1555c-322e-4368-836d-162700f5c6df service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:19:24.351946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1555c-322e-4368-836d-162700f5c6df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:19:24.353795 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-aeb1555c-322e-4368-836d-162700f5c6df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:19:24.367312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1555c-322e-4368-836d-162700f5c6df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:19:24.368772 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1555c-322e-4368-836d-162700f5c6df service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:19:24.380188 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1277/2559] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:19:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:19:25.490360 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f344c2-1031-4952-9d55-a1c31b924910 service nova] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:19:25.491825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-83f344c2-1031-4952-9d55-a1c31b924910 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:19:25.503193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f344c2-1031-4952-9d55-a1c31b924910 service nova] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:19:25.507503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be0670-a451-45eb-86fb-5639fe987ec7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:19:25.508092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be0670-a451-45eb-86fb-5639fe987ec7 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:19:25.508373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be0670-a451-45eb-86fb-5639fe987ec7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:19:25.508686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be0670-a451-45eb-86fb-5639fe987ec7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:19:25.510314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-79be0670-a451-45eb-86fb-5639fe987ec7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:19:25.523324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be0670-a451-45eb-86fb-5639fe987ec7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:19:25.524762 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be0670-a451-45eb-86fb-5639fe987ec7 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:19:25.537504 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1283/2560] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:19:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:19:27.040544 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1555c-322e-4368-836d-162700f5c6df service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:19:27.049944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1555c-322e-4368-836d-162700f5c6df service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:19:27.054557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239291ca-6a3b-47f4-92e1-d8a74db3e2da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:19:27.055148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239291ca-6a3b-47f4-92e1-d8a74db3e2da service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:19:27.055429 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239291ca-6a3b-47f4-92e1-d8a74db3e2da service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:19:27.055704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239291ca-6a3b-47f4-92e1-d8a74db3e2da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:19:27.057435 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-239291ca-6a3b-47f4-92e1-d8a74db3e2da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:19:27.238588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239291ca-6a3b-47f4-92e1-d8a74db3e2da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:19:27.240022 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239291ca-6a3b-47f4-92e1-d8a74db3e2da service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:19:27.248830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1278/2561] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:19:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:19:35.532383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=75 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:19:37.244882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=82 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:20:23.773682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be0670-a451-45eb-86fb-5639fe987ec7 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:23.791761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be0670-a451-45eb-86fb-5639fe987ec7 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:23.795232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852e630-26e0-4382-8f8e-4e6f28f74537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:23.795821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852e630-26e0-4382-8f8e-4e6f28f74537 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:23.796100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852e630-26e0-4382-8f8e-4e6f28f74537 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:23.796370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852e630-26e0-4382-8f8e-4e6f28f74537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:23.798103 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d852e630-26e0-4382-8f8e-4e6f28f74537 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:23.888217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852e630-26e0-4382-8f8e-4e6f28f74537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:23.889665 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852e630-26e0-4382-8f8e-4e6f28f74537 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:23.899782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1284/2562] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:20:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:20:27.112515 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239291ca-6a3b-47f4-92e1-d8a74db3e2da service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:27.112515 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-239291ca-6a3b-47f4-92e1-d8a74db3e2da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:27.154453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239291ca-6a3b-47f4-92e1-d8a74db3e2da service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:27.159331 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c411c3-fc98-41fb-965b-fd1535254cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:27.159968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c411c3-fc98-41fb-965b-fd1535254cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:27.160250 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c411c3-fc98-41fb-965b-fd1535254cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:27.160637 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c411c3-fc98-41fb-965b-fd1535254cb9 admin admin] PATH_INFO: `/v3/users/23a87870fdaf48b0b7b55193f41f8eb0` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:27.162693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c411c3-fc98-41fb-965b-fd1535254cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23a87870fdaf48b0b7b55193f41f8eb0)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:27.164744 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c411c3-fc98-41fb-965b-fd1535254cb9 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:27.344313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-36c411c3-fc98-41fb-965b-fd1535254cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23a87870fdaf48b0b7b55193f41f8eb0'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:27.414512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852e630-26e0-4382-8f8e-4e6f28f74537 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:27.416061 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d852e630-26e0-4382-8f8e-4e6f28f74537 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:27.416315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-36c411c3-fc98-41fb-965b-fd1535254cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23a87870fdaf48b0b7b55193f41f8eb0'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:27.417100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-36c411c3-fc98-41fb-965b-fd1535254cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23a87870fdaf48b0b7b55193f41f8eb0'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:27.425436 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-36c411c3-fc98-41fb-965b-fd1535254cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23a87870fdaf48b0b7b55193f41f8eb0'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:27.439980 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-36c411c3-fc98-41fb-965b-fd1535254cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23a87870fdaf48b0b7b55193f41f8eb0'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:27.447103 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1279/2563] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:20:27 2022] DELETE /identity/v3/users/23a87870fdaf48b0b7b55193f41f8eb0 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:20:27.452738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f4ede-24be-4f39-875b-5a740821b7e5 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:27.453029 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f4ede-24be-4f39-875b-5a740821b7e5 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:27.453323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f4ede-24be-4f39-875b-5a740821b7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:27.472032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-2e2f4ede-24be-4f39-875b-5a740821b7e5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:20:27.473252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2e2f4ede-24be-4f39-875b-5a740821b7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:27.525156 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852e630-26e0-4382-8f8e-4e6f28f74537 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:27.535299 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1280/2564] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:20:27.548230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f4ede-24be-4f39-875b-5a740821b7e5 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:27.549822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2e2f4ede-24be-4f39-875b-5a740821b7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:27.555807 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:20:27.592972 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407cf3b-e86e-410d-b594-bd87ea1c3ce9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:27.593739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407cf3b-e86e-410d-b594-bd87ea1c3ce9 service placement] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:27.594124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407cf3b-e86e-410d-b594-bd87ea1c3ce9 service placement] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:27.594474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407cf3b-e86e-410d-b594-bd87ea1c3ce9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:27.598425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e407cf3b-e86e-410d-b594-bd87ea1c3ce9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:27.606350 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:20:27.610998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407cf3b-e86e-410d-b594-bd87ea1c3ce9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:27.611557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f4ede-24be-4f39-875b-5a740821b7e5 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:27.612575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407cf3b-e86e-410d-b594-bd87ea1c3ce9 service placement] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:27.615395 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f1850-0903-4d99-9159-d1e3eac83f0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:27.615999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f1850-0903-4d99-9159-d1e3eac83f0e service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:27.616280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f1850-0903-4d99-9159-d1e3eac83f0e service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:27.616552 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f1850-0903-4d99-9159-d1e3eac83f0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:27.622434 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1285/2565] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:20:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:20:27.623302 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407cf3b-e86e-410d-b594-bd87ea1c3ce9 service placement] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:27.624612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e407cf3b-e86e-410d-b594-bd87ea1c3ce9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:27.626787 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f1850-0903-4d99-9159-d1e3eac83f0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:27.628225 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f1850-0903-4d99-9159-d1e3eac83f0e service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:27.638432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407cf3b-e86e-410d-b594-bd87ea1c3ce9 service placement] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:27.642677 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1281/2566] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:20:27.643644 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1119a2-f424-424f-97b8-864b395760c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:27.644245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1119a2-f424-424f-97b8-864b395760c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:27.644583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1119a2-f424-424f-97b8-864b395760c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:27.644889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1119a2-f424-424f-97b8-864b395760c0 admin admin] PATH_INFO: `/v3/users/2ec3daa674814619976aead75459e98c` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:27.655405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1119a2-f424-424f-97b8-864b395760c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ec3daa674814619976aead75459e98c)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:27.655405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1119a2-f424-424f-97b8-864b395760c0 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:27.829414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ea1119a2-f424-424f-97b8-864b395760c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ec3daa674814619976aead75459e98c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:27.840581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ea1119a2-f424-424f-97b8-864b395760c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ec3daa674814619976aead75459e98c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:27.889989 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ea1119a2-f424-424f-97b8-864b395760c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ec3daa674814619976aead75459e98c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:27.890789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ea1119a2-f424-424f-97b8-864b395760c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ec3daa674814619976aead75459e98c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:27.904838 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-ea1119a2-f424-424f-97b8-864b395760c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ec3daa674814619976aead75459e98c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:27.910754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1286/2567] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:20:27 2022] DELETE /identity/v3/users/2ec3daa674814619976aead75459e98c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:20:28.062700 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f1850-0903-4d99-9159-d1e3eac83f0e service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:28.066166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-d78f1850-0903-4d99-9159-d1e3eac83f0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:28.078625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f1850-0903-4d99-9159-d1e3eac83f0e service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:28.139077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963a582-0726-4ff4-a266-ba55b95331bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:28.139753 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963a582-0726-4ff4-a266-ba55b95331bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.140055 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963a582-0726-4ff4-a266-ba55b95331bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.140397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963a582-0726-4ff4-a266-ba55b95331bc admin admin] PATH_INFO: `/v3/projects/4123f99fac0349699118351a05998a96` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.141697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963a582-0726-4ff4-a266-ba55b95331bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4123f99fac0349699118351a05998a96)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:28.143664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963a582-0726-4ff4-a266-ba55b95331bc admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:28.194480 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1119a2-f424-424f-97b8-864b395760c0 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:28.196181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ea1119a2-f424-424f-97b8-864b395760c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:28.211508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1119a2-f424-424f-97b8-864b395760c0 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:28.216563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb602f4-0bf9-460c-a292-7194e4f5aca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:28.217297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb602f4-0bf9-460c-a292-7194e4f5aca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.217649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb602f4-0bf9-460c-a292-7194e4f5aca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.217998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb602f4-0bf9-460c-a292-7194e4f5aca8 admin admin] PATH_INFO: `/v3/projects/d7043fd37a1040c2af5d3f9f2943a791` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.219217 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb602f4-0bf9-460c-a292-7194e4f5aca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7043fd37a1040c2af5d3f9f2943a791)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:28.221300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb602f4-0bf9-460c-a292-7194e4f5aca8 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:28.267753 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a963a582-0726-4ff4-a266-ba55b95331bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4123f99fac0349699118351a05998a96', 'request_id': 'req-a963a582-0726-4ff4-a266-ba55b95331bc'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:28.274637 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a963a582-0726-4ff4-a266-ba55b95331bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4123f99fac0349699118351a05998a96', 'request_id': 'req-a963a582-0726-4ff4-a266-ba55b95331bc'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:28.313884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-afb602f4-0bf9-460c-a292-7194e4f5aca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7043fd37a1040c2af5d3f9f2943a791', 'request_id': 'req-afb602f4-0bf9-460c-a292-7194e4f5aca8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:28.320458 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-afb602f4-0bf9-460c-a292-7194e4f5aca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7043fd37a1040c2af5d3f9f2943a791', 'request_id': 'req-afb602f4-0bf9-460c-a292-7194e4f5aca8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:28.325920 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-afb602f4-0bf9-460c-a292-7194e4f5aca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7043fd37a1040c2af5d3f9f2943a791', 'request_id': 'req-afb602f4-0bf9-460c-a292-7194e4f5aca8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:28.353009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a963a582-0726-4ff4-a266-ba55b95331bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4123f99fac0349699118351a05998a96', 'request_id': 'req-a963a582-0726-4ff4-a266-ba55b95331bc'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:28.364004 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a963a582-0726-4ff4-a266-ba55b95331bc admin admin] The token cache is being invalidate because project 4123f99fac0349699118351a05998a96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:20:28.364295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a963a582-0726-4ff4-a266-ba55b95331bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:28.364942 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a963a582-0726-4ff4-a266-ba55b95331bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:28.366601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1282/2568] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:20:28 2022] DELETE /identity/v3/projects/4123f99fac0349699118351a05998a96 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:20:28.372836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352fe12-73dc-48e4-aeb1-c1740b0c0d29 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.373130 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352fe12-73dc-48e4-aeb1-c1740b0c0d29 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.373410 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352fe12-73dc-48e4-aeb1-c1740b0c0d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.393096 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-c352fe12-73dc-48e4-aeb1-c1740b0c0d29 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:20:28.394233 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c352fe12-73dc-48e4-aeb1-c1740b0c0d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:28.399033 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-afb602f4-0bf9-460c-a292-7194e4f5aca8 admin admin] The token cache is being invalidate because project d7043fd37a1040c2af5d3f9f2943a791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:20:28.400568 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-afb602f4-0bf9-460c-a292-7194e4f5aca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:28.400568 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-afb602f4-0bf9-460c-a292-7194e4f5aca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:20:28.405709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1287/2569] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:20:28 2022] DELETE /identity/v3/projects/d7043fd37a1040c2af5d3f9f2943a791 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:20:28.407567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc6cd0-8ac0-45c6-9738-ea6e5347afd4 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.407861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc6cd0-8ac0-45c6-9738-ea6e5347afd4 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.408142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc6cd0-8ac0-45c6-9738-ea6e5347afd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.428550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9dcc6cd0-8ac0-45c6-9738-ea6e5347afd4 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:20:28.429810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9dcc6cd0-8ac0-45c6-9738-ea6e5347afd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:28.438579 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:20:28.457302 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1283/2570] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:20:28.465692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeaea8-3cac-4397-a7d7-acba318cfd57 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.465982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeaea8-3cac-4397-a7d7-acba318cfd57 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.466259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeaea8-3cac-4397-a7d7-acba318cfd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.485997 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-cdeeaea8-3cac-4397-a7d7-acba318cfd57 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:20:28.487294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-cdeeaea8-3cac-4397-a7d7-acba318cfd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:28.493639 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1288/2571] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:20:28.496259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1284/2572] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:20:28.500579 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc6cd0-8ac0-45c6-9738-ea6e5347afd4 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:28.501451 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638a800-6af1-45e3-b38c-d4ffe5f61fc8 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.501736 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638a800-6af1-45e3-b38c-d4ffe5f61fc8 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.502011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638a800-6af1-45e3-b38c-d4ffe5f61fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.510774 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc6cd0-8ac0-45c6-9738-ea6e5347afd4 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:28.515406 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26c3ce-3db4-44f5-81f5-24882b9a3cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:28.516011 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26c3ce-3db4-44f5-81f5-24882b9a3cfa admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.516292 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26c3ce-3db4-44f5-81f5-24882b9a3cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.516569 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26c3ce-3db4-44f5-81f5-24882b9a3cfa admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.517246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26c3ce-3db4-44f5-81f5-24882b9a3cfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:20:28.517552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26c3ce-3db4-44f5-81f5-24882b9a3cfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:28.519155 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26c3ce-3db4-44f5-81f5-24882b9a3cfa admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:28.522257 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-5638a800-6af1-45e3-b38c-d4ffe5f61fc8 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:20:28.523561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5638a800-6af1-45e3-b38c-d4ffe5f61fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:28.531525 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1285/2573] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:20:28.533342 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1289/2574] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:20:28.535789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638a800-6af1-45e3-b38c-d4ffe5f61fc8 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:28.539243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26c3ce-3db4-44f5-81f5-24882b9a3cfa admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:28.545614 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638a800-6af1-45e3-b38c-d4ffe5f61fc8 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:28.549079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26c3ce-3db4-44f5-81f5-24882b9a3cfa admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:28.550225 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c83a53-6e12-4679-a836-0c7c7b2eac6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:28.550820 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c83a53-6e12-4679-a836-0c7c7b2eac6d admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.551100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c83a53-6e12-4679-a836-0c7c7b2eac6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.551373 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c83a53-6e12-4679-a836-0c7c7b2eac6d admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.552264 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c83a53-6e12-4679-a836-0c7c7b2eac6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:20:28.552565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c83a53-6e12-4679-a836-0c7c7b2eac6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:28.554003 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da002c-4230-46ee-b623-b78172d07da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:28.554401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c83a53-6e12-4679-a836-0c7c7b2eac6d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:28.554592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da002c-4230-46ee-b623-b78172d07da2 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.554878 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da002c-4230-46ee-b623-b78172d07da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.555150 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da002c-4230-46ee-b623-b78172d07da2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.555839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da002c-4230-46ee-b623-b78172d07da2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:28.555976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:20:28.557767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da002c-4230-46ee-b623-b78172d07da2 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:28.566175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1286/2575] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:20:28.570561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c83a53-6e12-4679-a836-0c7c7b2eac6d admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:28.579864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c83a53-6e12-4679-a836-0c7c7b2eac6d admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:28.585113 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c2465-2020-44c2-ba24-6aa3dfa028f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:28.585781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2465-2020-44c2-ba24-6aa3dfa028f7 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.586078 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2465-2020-44c2-ba24-6aa3dfa028f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.586345 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2465-2020-44c2-ba24-6aa3dfa028f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.586956 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c2465-2020-44c2-ba24-6aa3dfa028f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:28.588846 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c2465-2020-44c2-ba24-6aa3dfa028f7 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:28.645068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1290/2576] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:20:28 2022] POST /identity/v3/projects => generated 397 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:20:28.650037 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da002c-4230-46ee-b623-b78172d07da2 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:28.673293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1287/2577] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:20:28 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:20:28.674325 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c2465-2020-44c2-ba24-6aa3dfa028f7 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:28.714072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da002c-4230-46ee-b623-b78172d07da2 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:28.737320 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c2465-2020-44c2-ba24-6aa3dfa028f7 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:28.742073 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d049972-af3d-4c71-987f-7e830cc43ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:28.742669 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d049972-af3d-4c71-987f-7e830cc43ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.742947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d049972-af3d-4c71-987f-7e830cc43ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.743219 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d049972-af3d-4c71-987f-7e830cc43ca1 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.743834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d049972-af3d-4c71-987f-7e830cc43ca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:28.745773 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d049972-af3d-4c71-987f-7e830cc43ca1 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:28.773321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af5470-4f06-40d5-9be5-474179df442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:28.774079 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af5470-4f06-40d5-9be5-474179df442b admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.774209 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af5470-4f06-40d5-9be5-474179df442b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.774453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af5470-4f06-40d5-9be5-474179df442b admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.775072 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af5470-4f06-40d5-9be5-474179df442b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:28.777014 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af5470-4f06-40d5-9be5-474179df442b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:28.827375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1288/2578] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:20:28 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:20:28.831423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d049972-af3d-4c71-987f-7e830cc43ca1 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:28.841678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d049972-af3d-4c71-987f-7e830cc43ca1 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:28.846223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633db83c-7a65-4ff8-ac3e-2f65bbfa7a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:28.846787 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633db83c-7a65-4ff8-ac3e-2f65bbfa7a81 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.847065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633db83c-7a65-4ff8-ac3e-2f65bbfa7a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.847336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633db83c-7a65-4ff8-ac3e-2f65bbfa7a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.847954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633db83c-7a65-4ff8-ac3e-2f65bbfa7a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:28.849592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633db83c-7a65-4ff8-ac3e-2f65bbfa7a81 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:28.858880 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1291/2579] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:20:28 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:20:28.862345 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1289/2580] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:20:28.862517 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af5470-4f06-40d5-9be5-474179df442b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:28.863304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633db83c-7a65-4ff8-ac3e-2f65bbfa7a81 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:28.869718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af5470-4f06-40d5-9be5-474179df442b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:28.872831 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633db83c-7a65-4ff8-ac3e-2f65bbfa7a81 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:28.874397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388496d-128f-4fe8-8eaf-5e5c68d46d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:28.874981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388496d-128f-4fe8-8eaf-5e5c68d46d76 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.875267 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388496d-128f-4fe8-8eaf-5e5c68d46d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.875538 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388496d-128f-4fe8-8eaf-5e5c68d46d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.876201 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388496d-128f-4fe8-8eaf-5e5c68d46d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:28.877303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45d798-9b4d-4e0e-ba8b-e490c067782b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:28.877814 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388496d-128f-4fe8-8eaf-5e5c68d46d76 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:28.877955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45d798-9b4d-4e0e-ba8b-e490c067782b admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.878109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45d798-9b4d-4e0e-ba8b-e490c067782b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.878377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45d798-9b4d-4e0e-ba8b-e490c067782b admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.878998 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45d798-9b4d-4e0e-ba8b-e490c067782b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:28.880546 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45d798-9b4d-4e0e-ba8b-e490c067782b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:28.888770 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1292/2581] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:20:28.892523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1290/2582] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:20:28.892821 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388496d-128f-4fe8-8eaf-5e5c68d46d76 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:28.901189 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45d798-9b4d-4e0e-ba8b-e490c067782b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:28.901322 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:20:28.903453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388496d-128f-4fe8-8eaf-5e5c68d46d76 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:28.906743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45d798-9b4d-4e0e-ba8b-e490c067782b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:28.908717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c3e64-26a1-4fd0-a383-5ce4670fa85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:28.909325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c3e64-26a1-4fd0-a383-5ce4670fa85f admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.909660 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c3e64-26a1-4fd0-a383-5ce4670fa85f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.909874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c3e64-26a1-4fd0-a383-5ce4670fa85f admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.910519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c3e64-26a1-4fd0-a383-5ce4670fa85f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:28.911883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8121be-3408-46e9-917a-6337be72e4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:28.912127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c3e64-26a1-4fd0-a383-5ce4670fa85f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:28.912822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8121be-3408-46e9-917a-6337be72e4db admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.913174 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8121be-3408-46e9-917a-6337be72e4db admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.913512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8121be-3408-46e9-917a-6337be72e4db admin admin] PATH_INFO: `/v3/projects/0ed82d8ed9a4431f8528213ac5e91c7e/users/5fb0b775f9ee4a44a736346a173bdd49/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.923899 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1293/2583] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:20:28.929118 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c3e64-26a1-4fd0-a383-5ce4670fa85f admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:28.929260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8121be-3408-46e9-917a-6337be72e4db admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ed82d8ed9a4431f8528213ac5e91c7e, user_id=5fb0b775f9ee4a44a736346a173bdd49, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:28.931788 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8121be-3408-46e9-917a-6337be72e4db admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:28.938613 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c3e64-26a1-4fd0-a383-5ce4670fa85f admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:28.943995 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354e16e-c7b7-40eb-9b99-0f5e5d9b6bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:28.944712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354e16e-c7b7-40eb-9b99-0f5e5d9b6bba admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:28.945026 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354e16e-c7b7-40eb-9b99-0f5e5d9b6bba admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:28.945308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354e16e-c7b7-40eb-9b99-0f5e5d9b6bba admin admin] PATH_INFO: `/v3/projects/8d5bb084056f42a0a05fb9d5b20db778/users/b50338df1f67414bb25a3e965ca6f273/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:28.960129 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354e16e-c7b7-40eb-9b99-0f5e5d9b6bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d5bb084056f42a0a05fb9d5b20db778, user_id=b50338df1f67414bb25a3e965ca6f273, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:28.963540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354e16e-c7b7-40eb-9b99-0f5e5d9b6bba admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:29.005560 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1291/2584] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:20:28 2022] PUT /identity/v3/projects/0ed82d8ed9a4431f8528213ac5e91c7e/users/5fb0b775f9ee4a44a736346a173bdd49/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:20:29.016740 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42d649-7681-4b89-99ae-1f8db20670aa None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:29.017034 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42d649-7681-4b89-99ae-1f8db20670aa None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:29.017311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42d649-7681-4b89-99ae-1f8db20670aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:29.044717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-0f42d649-7681-4b89-99ae-1f8db20670aa None None] MFA Rules not processed for user `5fb0b775f9ee4a44a736346a173bdd49`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:20:29.045920 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0f42d649-7681-4b89-99ae-1f8db20670aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:29.066968 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1294/2585] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:20:28 2022] PUT /identity/v3/projects/8d5bb084056f42a0a05fb9d5b20db778/users/b50338df1f67414bb25a3e965ca6f273/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:20:29.070987 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a871fd7-49f9-4ef9-bb34-3f0419e6c5b7 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:29.071381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a871fd7-49f9-4ef9-bb34-3f0419e6c5b7 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:29.071721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a871fd7-49f9-4ef9-bb34-3f0419e6c5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:29.116980 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-5a871fd7-49f9-4ef9-bb34-3f0419e6c5b7 None None] MFA Rules not processed for user `b50338df1f67414bb25a3e965ca6f273`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:20:29.118321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5a871fd7-49f9-4ef9-bb34-3f0419e6c5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:29.128982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1292/2586] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:20:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:20:29.140616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f42d649-7681-4b89-99ae-1f8db20670aa None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:29.142035 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0f42d649-7681-4b89-99ae-1f8db20670aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:29.149545 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:20:29.191580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1295/2587] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:20:29 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:20:29.203991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a871fd7-49f9-4ef9-bb34-3f0419e6c5b7 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:29.205924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5a871fd7-49f9-4ef9-bb34-3f0419e6c5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:29.226896 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f42d649-7681-4b89-99ae-1f8db20670aa None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:29.236810 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5043fbe-61d5-468e-af91-bcc44a06c2f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:29.237417 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5043fbe-61d5-468e-af91-bcc44a06c2f2 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:29.238026 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5043fbe-61d5-468e-af91-bcc44a06c2f2 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:29.238026 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5043fbe-61d5-468e-af91-bcc44a06c2f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:29.246793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5043fbe-61d5-468e-af91-bcc44a06c2f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:29.248397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5043fbe-61d5-468e-af91-bcc44a06c2f2 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:29.257946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1293/2588] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:20:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:20:29.273394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a871fd7-49f9-4ef9-bb34-3f0419e6c5b7 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:29.277028 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128d552-f384-4ca6-81eb-328b11105128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:29.277625 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128d552-f384-4ca6-81eb-328b11105128 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:29.277902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128d552-f384-4ca6-81eb-328b11105128 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:29.278170 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128d552-f384-4ca6-81eb-328b11105128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:29.286248 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128d552-f384-4ca6-81eb-328b11105128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:29.287967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128d552-f384-4ca6-81eb-328b11105128 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:29.296474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1296/2589] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:20:29 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:20:32.287207 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5043fbe-61d5-468e-af91-bcc44a06c2f2 service nova] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:20:32.289197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c5043fbe-61d5-468e-af91-bcc44a06c2f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:32.357418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5043fbe-61d5-468e-af91-bcc44a06c2f2 service nova] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:20:32.362333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d3f99-8cdd-4883-a6a9-a88f1cc83068 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:20:32.362936 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d3f99-8cdd-4883-a6a9-a88f1cc83068 service placement] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:20:32.363215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d3f99-8cdd-4883-a6a9-a88f1cc83068 service placement] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:20:32.363487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d3f99-8cdd-4883-a6a9-a88f1cc83068 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:20:32.365311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f96d3f99-8cdd-4883-a6a9-a88f1cc83068 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:20:32.376784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d3f99-8cdd-4883-a6a9-a88f1cc83068 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:20:32.382059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d3f99-8cdd-4883-a6a9-a88f1cc83068 service placement] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:20:32.388579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1294/2590] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:20:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:20:39.292555 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=71,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:20:42.384556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=64 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:23:32.818074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128d552-f384-4ca6-81eb-328b11105128 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:23:32.820102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4128d552-f384-4ca6-81eb-328b11105128 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:23:32.841474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128d552-f384-4ca6-81eb-328b11105128 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:23:32.845107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de860d07-77c3-44d5-a914-387e599e7ba5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:23:32.845702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de860d07-77c3-44d5-a914-387e599e7ba5 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:23:32.846024 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de860d07-77c3-44d5-a914-387e599e7ba5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:23:32.846318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de860d07-77c3-44d5-a914-387e599e7ba5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:23:32.848020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-de860d07-77c3-44d5-a914-387e599e7ba5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:23:32.907138 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de860d07-77c3-44d5-a914-387e599e7ba5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:23:32.908586 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de860d07-77c3-44d5-a914-387e599e7ba5 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:23:32.917549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1297/2591] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:23:32 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:23:42.913433 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=32 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:24:23.382400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d3f99-8cdd-4883-a6a9-a88f1cc83068 service placement] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:24:23.391935 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d3f99-8cdd-4883-a6a9-a88f1cc83068 service placement] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:24:23.395557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43b6a3-862b-4a52-85c3-611f7412549c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:24:23.396171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b6a3-862b-4a52-85c3-611f7412549c service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:24:23.396507 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b6a3-862b-4a52-85c3-611f7412549c service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:24:23.396816 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b6a3-862b-4a52-85c3-611f7412549c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:24:23.398563 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-3c43b6a3-862b-4a52-85c3-611f7412549c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:24:23.457595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43b6a3-862b-4a52-85c3-611f7412549c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:24:23.459088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43b6a3-862b-4a52-85c3-611f7412549c service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:24:23.467996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1295/2592] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:24:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:24:25.667158 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de860d07-77c3-44d5-a914-387e599e7ba5 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:24:25.676096 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de860d07-77c3-44d5-a914-387e599e7ba5 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:24:25.679767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68931a-dde6-4ad6-99e9-0ea0034b8436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:24:25.680384 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68931a-dde6-4ad6-99e9-0ea0034b8436 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:24:25.680698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68931a-dde6-4ad6-99e9-0ea0034b8436 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:24:25.680980 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68931a-dde6-4ad6-99e9-0ea0034b8436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:24:25.682679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4e68931a-dde6-4ad6-99e9-0ea0034b8436 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:24:25.715808 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68931a-dde6-4ad6-99e9-0ea0034b8436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:24:25.717244 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68931a-dde6-4ad6-99e9-0ea0034b8436 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:24:25.737521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1298/2593] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:24:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:24:26.805101 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43b6a3-862b-4a52-85c3-611f7412549c service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:24:26.813628 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43b6a3-862b-4a52-85c3-611f7412549c service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:24:26.817183 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d846e10-dac5-4147-aa4b-95887873e785 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:24:26.817769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d846e10-dac5-4147-aa4b-95887873e785 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:24:26.818046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d846e10-dac5-4147-aa4b-95887873e785 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:24:26.818317 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d846e10-dac5-4147-aa4b-95887873e785 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:24:26.820251 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2d846e10-dac5-4147-aa4b-95887873e785 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:24:26.833018 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d846e10-dac5-4147-aa4b-95887873e785 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:24:26.834415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d846e10-dac5-4147-aa4b-95887873e785 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:24:26.845432 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1296/2594] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:24:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:24:35.735013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=18 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:24:36.840751 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=34 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:25:29.472080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68931a-dde6-4ad6-99e9-0ea0034b8436 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:25:29.480702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68931a-dde6-4ad6-99e9-0ea0034b8436 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:25:29.484301 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d151a53-d17e-4e8e-8e4a-3eb15ed6838f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:25:29.484939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d151a53-d17e-4e8e-8e4a-3eb15ed6838f service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:25:29.485221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d151a53-d17e-4e8e-8e4a-3eb15ed6838f service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:25:29.485494 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d151a53-d17e-4e8e-8e4a-3eb15ed6838f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:25:29.489262 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d846e10-dac5-4147-aa4b-95887873e785 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:25:29.490714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2d846e10-dac5-4147-aa4b-95887873e785 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:25:29.494651 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d151a53-d17e-4e8e-8e4a-3eb15ed6838f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:25:29.496070 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d151a53-d17e-4e8e-8e4a-3eb15ed6838f service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:25:29.505026 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d846e10-dac5-4147-aa4b-95887873e785 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:25:29.505238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1299/2595] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:25:29 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:25:29.511423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ae394-326d-4f1d-a349-b496ffd184c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:25:29.512025 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ae394-326d-4f1d-a349-b496ffd184c6 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:25:29.512310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ae394-326d-4f1d-a349-b496ffd184c6 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:25:29.512597 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ae394-326d-4f1d-a349-b496ffd184c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:25:29.520588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ae394-326d-4f1d-a349-b496ffd184c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:25:29.522190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ae394-326d-4f1d-a349-b496ffd184c6 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:25:29.531456 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1297/2596] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:25:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:25:38.256855 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d151a53-d17e-4e8e-8e4a-3eb15ed6838f service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:25:38.258921 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6d151a53-d17e-4e8e-8e4a-3eb15ed6838f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:25:38.273671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d151a53-d17e-4e8e-8e4a-3eb15ed6838f service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:25:38.278919 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff6a3d-fb93-41fd-b346-671ab8916da9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:25:38.279508 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff6a3d-fb93-41fd-b346-671ab8916da9 service placement] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:25:38.279787 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff6a3d-fb93-41fd-b346-671ab8916da9 service placement] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:25:38.280056 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff6a3d-fb93-41fd-b346-671ab8916da9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:25:38.288402 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff6a3d-fb93-41fd-b346-671ab8916da9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:25:38.290418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff6a3d-fb93-41fd-b346-671ab8916da9 service placement] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:25:38.300767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1300/2597] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:25:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:25:39.527690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=6 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:25:48.295907 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=12 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:26:30.762383 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ae394-326d-4f1d-a349-b496ffd184c6 service nova] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:30.844215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ae394-326d-4f1d-a349-b496ffd184c6 service nova] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:30.848923 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c862b-2bca-4976-aeb9-0016bac36e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:30.849546 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c862b-2bca-4976-aeb9-0016bac36e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:30.849824 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c862b-2bca-4976-aeb9-0016bac36e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:30.850135 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c862b-2bca-4976-aeb9-0016bac36e85 admin admin] PATH_INFO: `/v3/users/b50338df1f67414bb25a3e965ca6f273` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:30.851363 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c862b-2bca-4976-aeb9-0016bac36e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b50338df1f67414bb25a3e965ca6f273)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:30.854016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c862b-2bca-4976-aeb9-0016bac36e85 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:31.024059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-da3c862b-2bca-4976-aeb9-0016bac36e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b50338df1f67414bb25a3e965ca6f273'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:26:31.080289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-da3c862b-2bca-4976-aeb9-0016bac36e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b50338df1f67414bb25a3e965ca6f273'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:26:31.080937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-da3c862b-2bca-4976-aeb9-0016bac36e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b50338df1f67414bb25a3e965ca6f273'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:26:31.088615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-da3c862b-2bca-4976-aeb9-0016bac36e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b50338df1f67414bb25a3e965ca6f273'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:26:31.100297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-da3c862b-2bca-4976-aeb9-0016bac36e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b50338df1f67414bb25a3e965ca6f273'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:26:31.106327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1298/2598] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:26:30 2022] DELETE /identity/v3/users/b50338df1f67414bb25a3e965ca6f273 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:26:31.112317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bd477-aef8-4a01-8665-a2fee048a5f1 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:31.112865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bd477-aef8-4a01-8665-a2fee048a5f1 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:31.112981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bd477-aef8-4a01-8665-a2fee048a5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:31.152955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-201bd477-aef8-4a01-8665-a2fee048a5f1 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:26:31.154179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-201bd477-aef8-4a01-8665-a2fee048a5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:31.211402 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1301/2599] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:26:31.223084 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c862b-2bca-4976-aeb9-0016bac36e85 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:31.224471 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-da3c862b-2bca-4976-aeb9-0016bac36e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:31.299919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:26:31.309250 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c862b-2bca-4976-aeb9-0016bac36e85 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:31.313092 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb0406-e01b-4acc-89ef-a022c72486f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:31.313720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb0406-e01b-4acc-89ef-a022c72486f5 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:31.314004 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb0406-e01b-4acc-89ef-a022c72486f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:31.314280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb0406-e01b-4acc-89ef-a022c72486f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:31.323990 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb0406-e01b-4acc-89ef-a022c72486f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:31.325433 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb0406-e01b-4acc-89ef-a022c72486f5 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:31.335322 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1299/2600] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:26:31.716337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201bd477-aef8-4a01-8665-a2fee048a5f1 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:31.720181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-201bd477-aef8-4a01-8665-a2fee048a5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:31.730837 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201bd477-aef8-4a01-8665-a2fee048a5f1 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:31.735481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7599d-cf29-4fac-8859-72bea37c8e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:31.736107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7599d-cf29-4fac-8859-72bea37c8e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:31.736393 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7599d-cf29-4fac-8859-72bea37c8e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:31.736699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7599d-cf29-4fac-8859-72bea37c8e05 admin admin] PATH_INFO: `/v3/projects/8d5bb084056f42a0a05fb9d5b20db778` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:31.737841 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7599d-cf29-4fac-8859-72bea37c8e05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d5bb084056f42a0a05fb9d5b20db778)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:31.739759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7599d-cf29-4fac-8859-72bea37c8e05 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:31.851592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e3b7599d-cf29-4fac-8859-72bea37c8e05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d5bb084056f42a0a05fb9d5b20db778', 'request_id': 'req-e3b7599d-cf29-4fac-8859-72bea37c8e05'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:26:31.858380 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e3b7599d-cf29-4fac-8859-72bea37c8e05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d5bb084056f42a0a05fb9d5b20db778', 'request_id': 'req-e3b7599d-cf29-4fac-8859-72bea37c8e05'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:26:31.864068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e3b7599d-cf29-4fac-8859-72bea37c8e05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d5bb084056f42a0a05fb9d5b20db778', 'request_id': 'req-e3b7599d-cf29-4fac-8859-72bea37c8e05'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:26:31.932100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e3b7599d-cf29-4fac-8859-72bea37c8e05 admin admin] The token cache is being invalidate because project 8d5bb084056f42a0a05fb9d5b20db778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:26:31.932428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e3b7599d-cf29-4fac-8859-72bea37c8e05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:26:31.933095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-e3b7599d-cf29-4fac-8859-72bea37c8e05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:26:31.934729 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1302/2601] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:26:31 2022] DELETE /identity/v3/projects/8d5bb084056f42a0a05fb9d5b20db778 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:26:31.942183 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f393118-6a8e-4535-bae7-a9b694865647 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:31.942685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f393118-6a8e-4535-bae7-a9b694865647 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:31.942813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f393118-6a8e-4535-bae7-a9b694865647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:31.990008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-1f393118-6a8e-4535-bae7-a9b694865647 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:26:31.991221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-1f393118-6a8e-4535-bae7-a9b694865647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:32.049371 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1300/2602] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:26:32.058056 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03494d-0b3a-49fc-8c59-76c622bfac6e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:32.058345 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03494d-0b3a-49fc-8c59-76c622bfac6e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:32.058621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03494d-0b3a-49fc-8c59-76c622bfac6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:32.078619 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-2c03494d-0b3a-49fc-8c59-76c622bfac6e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:26:32.079848 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2c03494d-0b3a-49fc-8c59-76c622bfac6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:32.088086 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1303/2603] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:26:32.092288 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f393118-6a8e-4535-bae7-a9b694865647 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:32.101854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f393118-6a8e-4535-bae7-a9b694865647 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:32.108526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fbf7a-4540-4703-abb2-23fc3f4d1d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:32.109143 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fbf7a-4540-4703-abb2-23fc3f4d1d50 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:32.109561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fbf7a-4540-4703-abb2-23fc3f4d1d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:32.109740 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fbf7a-4540-4703-abb2-23fc3f4d1d50 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:32.110387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fbf7a-4540-4703-abb2-23fc3f4d1d50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:26:32.110686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fbf7a-4540-4703-abb2-23fc3f4d1d50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:32.112261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fbf7a-4540-4703-abb2-23fc3f4d1d50 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:32.123724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1301/2604] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:26:32.128052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03494d-0b3a-49fc-8c59-76c622bfac6e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:32.137378 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03494d-0b3a-49fc-8c59-76c622bfac6e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:32.141995 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13ddcf-41e9-474f-a421-1ab3c6af10b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:32.142580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13ddcf-41e9-474f-a421-1ab3c6af10b0 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:32.142857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13ddcf-41e9-474f-a421-1ab3c6af10b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:32.143128 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13ddcf-41e9-474f-a421-1ab3c6af10b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:32.143746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13ddcf-41e9-474f-a421-1ab3c6af10b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:32.145675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13ddcf-41e9-474f-a421-1ab3c6af10b0 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:32.230931 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1304/2605] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:26:32 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:26:32.235121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fbf7a-4540-4703-abb2-23fc3f4d1d50 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:32.245327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fbf7a-4540-4703-abb2-23fc3f4d1d50 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:32.304267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bc4f4-47d6-441a-85e3-f082a0719294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:32.304912 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bc4f4-47d6-441a-85e3-f082a0719294 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:32.305227 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bc4f4-47d6-441a-85e3-f082a0719294 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:32.305510 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bc4f4-47d6-441a-85e3-f082a0719294 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:32.306181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bc4f4-47d6-441a-85e3-f082a0719294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:32.308167 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bc4f4-47d6-441a-85e3-f082a0719294 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:32.366678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1302/2606] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:26:32 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:26:32.371006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13ddcf-41e9-474f-a421-1ab3c6af10b0 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:32.381167 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13ddcf-41e9-474f-a421-1ab3c6af10b0 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:32.385910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456208e7-2d4a-416c-9d28-6e0f2b1c9b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:32.390608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456208e7-2d4a-416c-9d28-6e0f2b1c9b7b admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:32.390608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456208e7-2d4a-416c-9d28-6e0f2b1c9b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:32.390608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456208e7-2d4a-416c-9d28-6e0f2b1c9b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:32.390608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456208e7-2d4a-416c-9d28-6e0f2b1c9b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:32.390608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456208e7-2d4a-416c-9d28-6e0f2b1c9b7b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:32.399142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1305/2607] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:26:32.403682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bc4f4-47d6-441a-85e3-f082a0719294 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:32.413585 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bc4f4-47d6-441a-85e3-f082a0719294 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:32.419149 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33cbd0-226d-4a78-9eaf-0baa11e8bf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:32.419738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33cbd0-226d-4a78-9eaf-0baa11e8bf77 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:32.420016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33cbd0-226d-4a78-9eaf-0baa11e8bf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:32.420285 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33cbd0-226d-4a78-9eaf-0baa11e8bf77 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:32.420953 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33cbd0-226d-4a78-9eaf-0baa11e8bf77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:32.422834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33cbd0-226d-4a78-9eaf-0baa11e8bf77 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:32.433658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1303/2608] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:26:32.437829 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456208e7-2d4a-416c-9d28-6e0f2b1c9b7b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:32.448431 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456208e7-2d4a-416c-9d28-6e0f2b1c9b7b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:32.453114 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c8a2b-daf9-4b21-822d-321f8a1482c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:32.453760 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c8a2b-daf9-4b21-822d-321f8a1482c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:32.454088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c8a2b-daf9-4b21-822d-321f8a1482c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:32.454349 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c8a2b-daf9-4b21-822d-321f8a1482c6 admin admin] PATH_INFO: `/v3/projects/19a004d608f2454cb5e9e65eef5ed740/users/b5600f62677d437ead57459e3cf75e41/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:32.469747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c8a2b-daf9-4b21-822d-321f8a1482c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a004d608f2454cb5e9e65eef5ed740, user_id=b5600f62677d437ead57459e3cf75e41, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:32.472417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c8a2b-daf9-4b21-822d-321f8a1482c6 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:32.474643 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:26:32.537228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1306/2609] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:26:32 2022] PUT /identity/v3/projects/19a004d608f2454cb5e9e65eef5ed740/users/b5600f62677d437ead57459e3cf75e41/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:26:32.542483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db0eff-6dd7-466a-86bd-f4bcaf69c165 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:32.542770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db0eff-6dd7-466a-86bd-f4bcaf69c165 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:32.543100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db0eff-6dd7-466a-86bd-f4bcaf69c165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:32.561981 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-54db0eff-6dd7-466a-86bd-f4bcaf69c165 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:26:32.563188 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-54db0eff-6dd7-466a-86bd-f4bcaf69c165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:32.583033 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:26:32.622124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1304/2610] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:26:32.631780 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c8a2b-daf9-4b21-822d-321f8a1482c6 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:32.633632 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-aa7c8a2b-daf9-4b21-822d-321f8a1482c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:32.690958 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c8a2b-daf9-4b21-822d-321f8a1482c6 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:32.694718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42203ea2-fff2-4dd7-b088-75fd4d3900df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:32.695318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42203ea2-fff2-4dd7-b088-75fd4d3900df service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:32.695598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42203ea2-fff2-4dd7-b088-75fd4d3900df service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:32.695869 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42203ea2-fff2-4dd7-b088-75fd4d3900df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:32.705679 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42203ea2-fff2-4dd7-b088-75fd4d3900df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:32.707101 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42203ea2-fff2-4dd7-b088-75fd4d3900df service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:32.716556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1307/2611] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:26:37.866141 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bcb5c-766b-4817-9e7e-502b327d50f1 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:37.866710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bcb5c-766b-4817-9e7e-502b327d50f1 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:37.866710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bcb5c-766b-4817-9e7e-502b327d50f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:37.897154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-155bcb5c-766b-4817-9e7e-502b327d50f1 None None] MFA Rules not processed for user `b5600f62677d437ead57459e3cf75e41`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:26:37.898441 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-155bcb5c-766b-4817-9e7e-502b327d50f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:37.970271 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1305/2612] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:26:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:26:37.974775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42203ea2-fff2-4dd7-b088-75fd4d3900df service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:37.984925 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42203ea2-fff2-4dd7-b088-75fd4d3900df service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:37.989571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36b5e6-e1c3-4d4b-94d5-de7b7e8a886b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:37.990182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36b5e6-e1c3-4d4b-94d5-de7b7e8a886b admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:37.990468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36b5e6-e1c3-4d4b-94d5-de7b7e8a886b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:37.990741 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36b5e6-e1c3-4d4b-94d5-de7b7e8a886b admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:37.991372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36b5e6-e1c3-4d4b-94d5-de7b7e8a886b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:37.993314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36b5e6-e1c3-4d4b-94d5-de7b7e8a886b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:38.111637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1308/2613] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:26:37 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:26:38.116114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bcb5c-766b-4817-9e7e-502b327d50f1 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:38.134011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bcb5c-766b-4817-9e7e-502b327d50f1 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:38.195217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7e1af-4fa4-4805-87fe-6ffd091e481b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:38.195828 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7e1af-4fa4-4805-87fe-6ffd091e481b admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:38.196108 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7e1af-4fa4-4805-87fe-6ffd091e481b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:38.196381 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7e1af-4fa4-4805-87fe-6ffd091e481b admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:38.197058 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7e1af-4fa4-4805-87fe-6ffd091e481b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:38.199979 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7e1af-4fa4-4805-87fe-6ffd091e481b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:38.313854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1306/2614] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:26:38 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:26:38.317491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36b5e6-e1c3-4d4b-94d5-de7b7e8a886b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:38.332913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36b5e6-e1c3-4d4b-94d5-de7b7e8a886b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:38.341631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e907bb-93f0-4ab9-a2b9-6f4ac149a5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:38.341631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e907bb-93f0-4ab9-a2b9-6f4ac149a5af admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:38.341631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e907bb-93f0-4ab9-a2b9-6f4ac149a5af admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:38.341631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e907bb-93f0-4ab9-a2b9-6f4ac149a5af admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:38.342049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e907bb-93f0-4ab9-a2b9-6f4ac149a5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:38.342049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e907bb-93f0-4ab9-a2b9-6f4ac149a5af admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:38.352589 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1309/2615] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:26:38.357145 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7e1af-4fa4-4805-87fe-6ffd091e481b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:38.368409 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7e1af-4fa4-4805-87fe-6ffd091e481b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:38.373977 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea3b6a-8d80-4997-ab83-e6139a10ffc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:38.374636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea3b6a-8d80-4997-ab83-e6139a10ffc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:38.374951 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea3b6a-8d80-4997-ab83-e6139a10ffc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:38.375300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea3b6a-8d80-4997-ab83-e6139a10ffc7 admin admin] PATH_INFO: `/v3/projects/3c8624cf5bb7465f8214347ac02be54e/users/2a55fd90c7c54f5bb842707b84a4ed9c/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:38.391925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea3b6a-8d80-4997-ab83-e6139a10ffc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c8624cf5bb7465f8214347ac02be54e, user_id=2a55fd90c7c54f5bb842707b84a4ed9c, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:38.394653 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea3b6a-8d80-4997-ab83-e6139a10ffc7 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:38.470794 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1307/2616] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:26:38 2022] PUT /identity/v3/projects/3c8624cf5bb7465f8214347ac02be54e/users/2a55fd90c7c54f5bb842707b84a4ed9c/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:26:38.474829 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e907bb-93f0-4ab9-a2b9-6f4ac149a5af admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:38.486034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e907bb-93f0-4ab9-a2b9-6f4ac149a5af admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:38.543824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eaa475-b079-4db1-868f-b88c3ae376b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:38.544376 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eaa475-b079-4db1-868f-b88c3ae376b9 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:38.544722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eaa475-b079-4db1-868f-b88c3ae376b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:38.544987 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eaa475-b079-4db1-868f-b88c3ae376b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:38.545630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eaa475-b079-4db1-868f-b88c3ae376b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:38.547503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eaa475-b079-4db1-868f-b88c3ae376b9 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:38.558444 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1310/2617] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:26:38.562620 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea3b6a-8d80-4997-ab83-e6139a10ffc7 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:38.572988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea3b6a-8d80-4997-ab83-e6139a10ffc7 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:38.577675 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195b3a9-f976-43c1-b54a-27b1219404e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:38.578319 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195b3a9-f976-43c1-b54a-27b1219404e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:38.578604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195b3a9-f976-43c1-b54a-27b1219404e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:38.578888 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195b3a9-f976-43c1-b54a-27b1219404e5 admin admin] PATH_INFO: `/v3/projects/3c8624cf5bb7465f8214347ac02be54e/users/2a55fd90c7c54f5bb842707b84a4ed9c/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:38.580830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195b3a9-f976-43c1-b54a-27b1219404e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c8624cf5bb7465f8214347ac02be54e, user_id=2a55fd90c7c54f5bb842707b84a4ed9c, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:38.583486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195b3a9-f976-43c1-b54a-27b1219404e5 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:38.593737 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1308/2618] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:26:38 2022] PUT /identity/v3/projects/3c8624cf5bb7465f8214347ac02be54e/users/2a55fd90c7c54f5bb842707b84a4ed9c/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:26:44.199622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fc12a-8607-4f23-bded-fc2565f5024d None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:44.200192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fc12a-8607-4f23-bded-fc2565f5024d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:44.200300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fc12a-8607-4f23-bded-fc2565f5024d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:44.233276 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-4d1fc12a-8607-4f23-bded-fc2565f5024d None None] MFA Rules not processed for user `2a55fd90c7c54f5bb842707b84a4ed9c`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:26:44.234555 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-4d1fc12a-8607-4f23-bded-fc2565f5024d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:44.294039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:26:44.324804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1311/2619] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:26:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:26:44.336725 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195b3a9-f976-43c1-b54a-27b1219404e5 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:44.344187 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-1195b3a9-f976-43c1-b54a-27b1219404e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:44.363995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:26:44.445837 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195b3a9-f976-43c1-b54a-27b1219404e5 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:44.450721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43bba4-6645-4ec9-b880-6425de1a342e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38de805ba5eb4aa58cf95d2082c2d216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:44.451887 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43bba4-6645-4ec9-b880-6425de1a342e service glance] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:44.452173 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43bba4-6645-4ec9-b880-6425de1a342e service glance] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:44.452446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43bba4-6645-4ec9-b880-6425de1a342e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:44.513594 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43bba4-6645-4ec9-b880-6425de1a342e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:44.515077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43bba4-6645-4ec9-b880-6425de1a342e service glance] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:44.524019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1309/2620] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:26:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:26:44.782139 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763a6f5-a2cb-4a7e-aeb8-6dadb8008dd1 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:44.782439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763a6f5-a2cb-4a7e-aeb8-6dadb8008dd1 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:44.782722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763a6f5-a2cb-4a7e-aeb8-6dadb8008dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:44.857955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0763a6f5-a2cb-4a7e-aeb8-6dadb8008dd1 None None] MFA Rules not processed for user `be379e1a6add42b69e9fdd676efca280`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:26:44.859255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0763a6f5-a2cb-4a7e-aeb8-6dadb8008dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:44.938558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1312/2621] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 24 08:26:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:26:45.078881 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43bba4-6645-4ec9-b880-6425de1a342e service glance] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:45.080351 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ad43bba4-6645-4ec9-b880-6425de1a342e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:45.176858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43bba4-6645-4ec9-b880-6425de1a342e service glance] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:45.182808 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d46dd-5599-4ec3-8a17-e4f41c61b382 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b629ad3e6b9f4723b103f9845df2aaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:45.183667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d46dd-5599-4ec3-8a17-e4f41c61b382 service swift] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:45.184024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d46dd-5599-4ec3-8a17-e4f41c61b382 service swift] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:45.184375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d46dd-5599-4ec3-8a17-e4f41c61b382 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:45.192909 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d46dd-5599-4ec3-8a17-e4f41c61b382 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:45.194509 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d46dd-5599-4ec3-8a17-e4f41c61b382 service swift] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:45.202964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1310/2622] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 24 08:26:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:26:54.935592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=51 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:26:55.199851 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=94 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:26:55.462142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a5664-8699-4320-a6e1-1cacb624371d None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:55.462465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a5664-8699-4320-a6e1-1cacb624371d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:55.462765 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a5664-8699-4320-a6e1-1cacb624371d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:55.481909 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-757a5664-8699-4320-a6e1-1cacb624371d None None] MFA Rules not processed for user `b5600f62677d437ead57459e3cf75e41`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:26:55.483128 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-757a5664-8699-4320-a6e1-1cacb624371d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:55.490000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1313/2623] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:26:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:26:55.746787 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d46dd-5599-4ec3-8a17-e4f41c61b382 service swift] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:55.810356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d46dd-5599-4ec3-8a17-e4f41c61b382 service swift] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:55.814280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a402561-badf-4f1a-a9ed-1d6061c59c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:55.814883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a402561-badf-4f1a-a9ed-1d6061c59c4d service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:55.815165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a402561-badf-4f1a-a9ed-1d6061c59c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:55.815440 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a402561-badf-4f1a-a9ed-1d6061c59c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:55.817239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6a402561-badf-4f1a-a9ed-1d6061c59c4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:55.826944 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a402561-badf-4f1a-a9ed-1d6061c59c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:55.828377 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a402561-badf-4f1a-a9ed-1d6061c59c4d service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:55.836419 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1311/2624] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:26:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:26:57.267380 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a5664-8699-4320-a6e1-1cacb624371d None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:26:57.268805 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-757a5664-8699-4320-a6e1-1cacb624371d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:57.340335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a5664-8699-4320-a6e1-1cacb624371d None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:26:57.345426 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dca5e3-b45d-4659-8bbe-c75a4300cc9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:26:57.346051 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dca5e3-b45d-4659-8bbe-c75a4300cc9a service placement] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:26:57.346334 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dca5e3-b45d-4659-8bbe-c75a4300cc9a service placement] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:26:57.346610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dca5e3-b45d-4659-8bbe-c75a4300cc9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:26:57.348368 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-89dca5e3-b45d-4659-8bbe-c75a4300cc9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:26:57.360884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dca5e3-b45d-4659-8bbe-c75a4300cc9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:26:57.362478 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dca5e3-b45d-4659-8bbe-c75a4300cc9a service placement] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:26:57.373306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1314/2625] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:26:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:27:02.931587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cdfe5-c678-4627-a8b0-8dddb84ec1b3 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:27:02.932049 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cdfe5-c678-4627-a8b0-8dddb84ec1b3 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:27:02.932154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cdfe5-c678-4627-a8b0-8dddb84ec1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:27:02.952764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-238cdfe5-c678-4627-a8b0-8dddb84ec1b3 None None] MFA Rules not processed for user `be379e1a6add42b69e9fdd676efca280`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:27:02.954564 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-238cdfe5-c678-4627-a8b0-8dddb84ec1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:27:02.961345 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1312/2626] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 24 08:27:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:27:02.973664 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dca5e3-b45d-4659-8bbe-c75a4300cc9a service placement] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:27:02.985424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dca5e3-b45d-4659-8bbe-c75a4300cc9a service placement] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:27:02.990707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6b90d-654f-49fd-a90e-495fa361dc1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b629ad3e6b9f4723b103f9845df2aaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:27:02.991381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6b90d-654f-49fd-a90e-495fa361dc1f service swift] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:27:02.991737 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6b90d-654f-49fd-a90e-495fa361dc1f service swift] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:27:02.992086 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6b90d-654f-49fd-a90e-495fa361dc1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:27:03.001571 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6b90d-654f-49fd-a90e-495fa361dc1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:27:03.003231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6b90d-654f-49fd-a90e-495fa361dc1f service swift] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:27:03.011703 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1315/2627] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 24 08:27:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:27:03.381158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cdfe5-c678-4627-a8b0-8dddb84ec1b3 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:27:03.382952 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-238cdfe5-c678-4627-a8b0-8dddb84ec1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:27:03.404246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cdfe5-c678-4627-a8b0-8dddb84ec1b3 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:27:03.415682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0303ede-a040-4bc1-8d4e-92103a482ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:27:03.416389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0303ede-a040-4bc1-8d4e-92103a482ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:27:03.416864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0303ede-a040-4bc1-8d4e-92103a482ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:27:03.417005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0303ede-a040-4bc1-8d4e-92103a482ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:27:03.421792 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f0303ede-a040-4bc1-8d4e-92103a482ac7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:27:03.460682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0303ede-a040-4bc1-8d4e-92103a482ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:27:03.468429 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0303ede-a040-4bc1-8d4e-92103a482ac7 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:27:03.517065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1313/2628] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:27:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:27:04.475964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6b90d-654f-49fd-a90e-495fa361dc1f service swift] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:27:04.477778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-44c6b90d-654f-49fd-a90e-495fa361dc1f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:27:04.645030 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6b90d-654f-49fd-a90e-495fa361dc1f service swift] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:27:04.652285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fa74d-ebb5-445b-baaf-7b8fcbaf5dca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:27:04.652949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fa74d-ebb5-445b-baaf-7b8fcbaf5dca service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:27:04.653266 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fa74d-ebb5-445b-baaf-7b8fcbaf5dca service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:27:04.653595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fa74d-ebb5-445b-baaf-7b8fcbaf5dca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:27:04.655618 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-317fa74d-ebb5-445b-baaf-7b8fcbaf5dca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:27:04.673580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317fa74d-ebb5-445b-baaf-7b8fcbaf5dca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:27:04.676218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317fa74d-ebb5-445b-baaf-7b8fcbaf5dca service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:27:04.693075 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1316/2629] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:27:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:27:13.491702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=35 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:27:14.686215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=75 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:27:18.885899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0303ede-a040-4bc1-8d4e-92103a482ac7 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:27:18.887812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f0303ede-a040-4bc1-8d4e-92103a482ac7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:27:18.904313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0303ede-a040-4bc1-8d4e-92103a482ac7 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:27:18.909915 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015e3f6-b7f5-4b19-af85-fb7c9adf52ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:27:18.913786 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015e3f6-b7f5-4b19-af85-fb7c9adf52ee service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:27:18.913786 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015e3f6-b7f5-4b19-af85-fb7c9adf52ee service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:27:18.913786 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015e3f6-b7f5-4b19-af85-fb7c9adf52ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:27:18.913786 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-a015e3f6-b7f5-4b19-af85-fb7c9adf52ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:27:18.924589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015e3f6-b7f5-4b19-af85-fb7c9adf52ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:27:18.926171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015e3f6-b7f5-4b19-af85-fb7c9adf52ee service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:27:18.936291 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1314/2630] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:27:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:27:28.932020 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=6 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:28:21.161466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fa74d-ebb5-445b-baaf-7b8fcbaf5dca service nova] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:21.163075 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-317fa74d-ebb5-445b-baaf-7b8fcbaf5dca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:21.243523 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fa74d-ebb5-445b-baaf-7b8fcbaf5dca service nova] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:21.247246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699d4ee-399f-4bef-bbb8-61f13ef6a131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:21.247842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699d4ee-399f-4bef-bbb8-61f13ef6a131 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:21.248119 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699d4ee-399f-4bef-bbb8-61f13ef6a131 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:21.248389 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699d4ee-399f-4bef-bbb8-61f13ef6a131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:21.250142 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6699d4ee-399f-4bef-bbb8-61f13ef6a131 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:21.311649 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699d4ee-399f-4bef-bbb8-61f13ef6a131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:21.313132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699d4ee-399f-4bef-bbb8-61f13ef6a131 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:21.322416 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1317/2631] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:28:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:28:31.318127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=55 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:28:32.306785 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015e3f6-b7f5-4b19-af85-fb7c9adf52ee service nova] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:32.316897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015e3f6-b7f5-4b19-af85-fb7c9adf52ee service nova] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:32.320602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf1ad4-28db-436c-a380-959a6ed410c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:32.321221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf1ad4-28db-436c-a380-959a6ed410c2 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:32.321504 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf1ad4-28db-436c-a380-959a6ed410c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:32.325736 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf1ad4-28db-436c-a380-959a6ed410c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:32.332786 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf1ad4-28db-436c-a380-959a6ed410c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:32.334264 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf1ad4-28db-436c-a380-959a6ed410c2 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:32.344364 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1315/2632] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:28:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:28:32.861576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699d4ee-399f-4bef-bbb8-61f13ef6a131 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:32.863128 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6699d4ee-399f-4bef-bbb8-61f13ef6a131 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:32.874830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699d4ee-399f-4bef-bbb8-61f13ef6a131 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:32.879246 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cc401-69e0-4140-b32f-287edf5c0b75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:32.879877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cc401-69e0-4140-b32f-287edf5c0b75 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:32.880184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cc401-69e0-4140-b32f-287edf5c0b75 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:32.880451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cc401-69e0-4140-b32f-287edf5c0b75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:32.882209 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b86cc401-69e0-4140-b32f-287edf5c0b75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:32.983619 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cc401-69e0-4140-b32f-287edf5c0b75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:32.988078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cc401-69e0-4140-b32f-287edf5c0b75 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:33.004512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1318/2633] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:28:32 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:28:33.496907 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf1ad4-28db-436c-a380-959a6ed410c2 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:33.498401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-aeaf1ad4-28db-436c-a380-959a6ed410c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:33.571309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf1ad4-28db-436c-a380-959a6ed410c2 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:33.576418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92776c-85cf-4849-9b0e-2259396f96d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:33.577154 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92776c-85cf-4849-9b0e-2259396f96d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:33.577541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92776c-85cf-4849-9b0e-2259396f96d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:33.577821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92776c-85cf-4849-9b0e-2259396f96d2 admin admin] PATH_INFO: `/v3/users/24273cc04c614d13ae47048a0f0ff5b1` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:33.578933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92776c-85cf-4849-9b0e-2259396f96d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24273cc04c614d13ae47048a0f0ff5b1)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:33.580949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92776c-85cf-4849-9b0e-2259396f96d2 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:33.757026 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-3a92776c-85cf-4849-9b0e-2259396f96d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24273cc04c614d13ae47048a0f0ff5b1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:33.850483 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-3a92776c-85cf-4849-9b0e-2259396f96d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24273cc04c614d13ae47048a0f0ff5b1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:33.851169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-3a92776c-85cf-4849-9b0e-2259396f96d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24273cc04c614d13ae47048a0f0ff5b1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:33.862880 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-3a92776c-85cf-4849-9b0e-2259396f96d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24273cc04c614d13ae47048a0f0ff5b1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:33.877356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-3a92776c-85cf-4849-9b0e-2259396f96d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24273cc04c614d13ae47048a0f0ff5b1'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:33.889293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1316/2634] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:28:33 2022] DELETE /identity/v3/users/24273cc04c614d13ae47048a0f0ff5b1 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:28:33.891213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d9fbe-c76c-463e-9ede-539c27cf218a None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:33.891674 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d9fbe-c76c-463e-9ede-539c27cf218a None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:33.891811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d9fbe-c76c-463e-9ede-539c27cf218a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:33.911800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-509d9fbe-c76c-463e-9ede-539c27cf218a None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:28:33.916510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-509d9fbe-c76c-463e-9ede-539c27cf218a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:33.978727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1319/2635] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:28:33.990636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92776c-85cf-4849-9b0e-2259396f96d2 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:33.992181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-3a92776c-85cf-4849-9b0e-2259396f96d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:34.060988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92776c-85cf-4849-9b0e-2259396f96d2 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:34.064674 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761be195-77aa-4279-ab4d-83885ba22de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:34.065274 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761be195-77aa-4279-ab4d-83885ba22de4 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:34.069847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761be195-77aa-4279-ab4d-83885ba22de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:34.069847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761be195-77aa-4279-ab4d-83885ba22de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:34.074714 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761be195-77aa-4279-ab4d-83885ba22de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:34.076205 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761be195-77aa-4279-ab4d-83885ba22de4 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:34.085950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1317/2636] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:28:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:28:34.777517 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d9fbe-c76c-463e-9ede-539c27cf218a None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:34.779602 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-509d9fbe-c76c-463e-9ede-539c27cf218a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:34.793898 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d9fbe-c76c-463e-9ede-539c27cf218a None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:34.800059 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:34.803108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:34.803108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:34.803108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] PATH_INFO: `/v3/projects/f5f23b4842fa4544bf28efeaa21c9ed3` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:34.803108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5f23b4842fa4544bf28efeaa21c9ed3)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:34.837914 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:34.962482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f23b4842fa4544bf28efeaa21c9ed3', 'request_id': 'req-beac0cf7-d514-409e-b93e-097ec3e40ea8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:34.969775 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f23b4842fa4544bf28efeaa21c9ed3', 'request_id': 'req-beac0cf7-d514-409e-b93e-097ec3e40ea8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:34.977567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f23b4842fa4544bf28efeaa21c9ed3', 'request_id': 'req-beac0cf7-d514-409e-b93e-097ec3e40ea8'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:34.985335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:28:35.088552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] The token cache is being invalidate because project f5f23b4842fa4544bf28efeaa21c9ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:28:35.089089 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:35.093529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:35.096749 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1320/2637] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:28:34 2022] DELETE /identity/v3/projects/f5f23b4842fa4544bf28efeaa21c9ed3 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:28:35.100470 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761be195-77aa-4279-ab4d-83885ba22de4 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:35.116975 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761be195-77aa-4279-ab4d-83885ba22de4 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:35.131555 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:28:35.202268 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6b113-9a9c-4de9-97f7-43595da065f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:35.203147 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6b113-9a9c-4de9-97f7-43595da065f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:35.203883 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6b113-9a9c-4de9-97f7-43595da065f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:35.204394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6b113-9a9c-4de9-97f7-43595da065f9 admin admin] PATH_INFO: `/v3/users/c2b3ff7e29024529913f4a894ac655e8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:35.226088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6b113-9a9c-4de9-97f7-43595da065f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2b3ff7e29024529913f4a894ac655e8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:35.228601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6b113-9a9c-4de9-97f7-43595da065f9 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:35.434604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-0af6b113-9a9c-4de9-97f7-43595da065f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2b3ff7e29024529913f4a894ac655e8'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:35.523150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-0af6b113-9a9c-4de9-97f7-43595da065f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2b3ff7e29024529913f4a894ac655e8'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:35.523775 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-0af6b113-9a9c-4de9-97f7-43595da065f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2b3ff7e29024529913f4a894ac655e8'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:35.532306 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-0af6b113-9a9c-4de9-97f7-43595da065f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2b3ff7e29024529913f4a894ac655e8'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:35.544611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-0af6b113-9a9c-4de9-97f7-43595da065f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2b3ff7e29024529913f4a894ac655e8'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:35.551829 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1318/2638] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:28:35 2022] DELETE /identity/v3/users/c2b3ff7e29024529913f4a894ac655e8 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:28:36.064769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:36.066369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:36.095720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac0cf7-d514-409e-b93e-097ec3e40ea8 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:36.154381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:36.155184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:36.155472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:36.155753 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] PATH_INFO: `/v3/projects/79816b997700419e84c65b766bfa343a` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:36.169298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79816b997700419e84c65b766bfa343a)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:36.171982 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:36.302313 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79816b997700419e84c65b766bfa343a', 'request_id': 'req-44246e06-9922-42ef-a481-cb969f01c821'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:36.308999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79816b997700419e84c65b766bfa343a', 'request_id': 'req-44246e06-9922-42ef-a481-cb969f01c821'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:36.315094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79816b997700419e84c65b766bfa343a', 'request_id': 'req-44246e06-9922-42ef-a481-cb969f01c821'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:36.491718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] The token cache is being invalidate because project 79816b997700419e84c65b766bfa343a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:28:36.492024 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:36.492740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:36.494684 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1321/2639] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:28:36 2022] DELETE /identity/v3/projects/79816b997700419e84c65b766bfa343a => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:28:36.499806 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318dde6-1f65-49ba-bb96-42bebb2bd53c None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:36.500310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318dde6-1f65-49ba-bb96-42bebb2bd53c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:36.500496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318dde6-1f65-49ba-bb96-42bebb2bd53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:36.512874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:36.515287 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:36.523713 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-f318dde6-1f65-49ba-bb96-42bebb2bd53c None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:28:36.524957 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f318dde6-1f65-49ba-bb96-42bebb2bd53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:36.579938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44246e06-9922-42ef-a481-cb969f01c821 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:36.583749 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95910b-7831-45a9-9249-9406dc1edd16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:36.584517 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95910b-7831-45a9-9249-9406dc1edd16 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:36.584898 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95910b-7831-45a9-9249-9406dc1edd16 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:36.585260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95910b-7831-45a9-9249-9406dc1edd16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:36.587055 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2c95910b-7831-45a9-9249-9406dc1edd16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:36.599824 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1319/2640] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:28:36.609663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a308c-28f2-46e5-bec4-ab50b3aeb86a None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:36.609879 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a308c-28f2-46e5-bec4-ab50b3aeb86a None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:36.610158 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a308c-28f2-46e5-bec4-ab50b3aeb86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:36.631994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-2d9a308c-28f2-46e5-bec4-ab50b3aeb86a None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:28:36.633523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2d9a308c-28f2-46e5-bec4-ab50b3aeb86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:36.646281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1320/2641] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:28:36.647431 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a308c-28f2-46e5-bec4-ab50b3aeb86a None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:36.662747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a308c-28f2-46e5-bec4-ab50b3aeb86a None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:36.668261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f4a1e-fffe-450e-92d6-41ae86bc188c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:36.668951 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f4a1e-fffe-450e-92d6-41ae86bc188c admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:36.669228 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f4a1e-fffe-450e-92d6-41ae86bc188c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:36.669495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f4a1e-fffe-450e-92d6-41ae86bc188c admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:36.671180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f4a1e-fffe-450e-92d6-41ae86bc188c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:28:36.671487 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f4a1e-fffe-450e-92d6-41ae86bc188c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:36.673252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f4a1e-fffe-450e-92d6-41ae86bc188c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:36.681447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:28:36.687898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1321/2642] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:28:36.693426 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f4a1e-fffe-450e-92d6-41ae86bc188c admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:36.703770 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f4a1e-fffe-450e-92d6-41ae86bc188c admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:36.708575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7112046-0e88-429b-8a95-af159b72eba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:36.709289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7112046-0e88-429b-8a95-af159b72eba0 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:36.709568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7112046-0e88-429b-8a95-af159b72eba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:36.710027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7112046-0e88-429b-8a95-af159b72eba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:36.710590 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7112046-0e88-429b-8a95-af159b72eba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:36.712802 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7112046-0e88-429b-8a95-af159b72eba0 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:36.750831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:28:36.784551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95910b-7831-45a9-9249-9406dc1edd16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:36.786019 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95910b-7831-45a9-9249-9406dc1edd16 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:36.796566 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1322/2643] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:28:36.816095 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1322/2644] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:28:36 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:28:36.820369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95910b-7831-45a9-9249-9406dc1edd16 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:36.829910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95910b-7831-45a9-9249-9406dc1edd16 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:36.903904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72d1fb-2123-4e96-a5fb-5164b39791aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:36.905180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72d1fb-2123-4e96-a5fb-5164b39791aa admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:36.905593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72d1fb-2123-4e96-a5fb-5164b39791aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:36.905972 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72d1fb-2123-4e96-a5fb-5164b39791aa admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:36.906677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72d1fb-2123-4e96-a5fb-5164b39791aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:36.908704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72d1fb-2123-4e96-a5fb-5164b39791aa admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:37.021319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1323/2645] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:28:36 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:28:37.025517 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7112046-0e88-429b-8a95-af159b72eba0 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:37.036546 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7112046-0e88-429b-8a95-af159b72eba0 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:37.041409 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5e0df-33a0-4850-b50e-9a92002f6f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:37.042079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5e0df-33a0-4850-b50e-9a92002f6f0d admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:37.042431 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5e0df-33a0-4850-b50e-9a92002f6f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:37.042790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5e0df-33a0-4850-b50e-9a92002f6f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:37.043601 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5e0df-33a0-4850-b50e-9a92002f6f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:37.046164 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5e0df-33a0-4850-b50e-9a92002f6f0d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:37.057129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1323/2646] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:28:37.061440 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72d1fb-2123-4e96-a5fb-5164b39791aa admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:37.071629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72d1fb-2123-4e96-a5fb-5164b39791aa admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:37.076595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5aa1e-fb51-41c3-88e5-b2bfc6f14955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:37.077333 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5aa1e-fb51-41c3-88e5-b2bfc6f14955 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:37.077681 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5aa1e-fb51-41c3-88e5-b2bfc6f14955 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:37.077961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5aa1e-fb51-41c3-88e5-b2bfc6f14955 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:37.078645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5aa1e-fb51-41c3-88e5-b2bfc6f14955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:37.080978 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5aa1e-fb51-41c3-88e5-b2bfc6f14955 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:37.097291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1324/2647] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:28:37.098987 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5e0df-33a0-4850-b50e-9a92002f6f0d admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:37.112248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5e0df-33a0-4850-b50e-9a92002f6f0d admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:37.117827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cd693-97db-48b2-a564-2bb58f60e038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:37.119367 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cd693-97db-48b2-a564-2bb58f60e038 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:37.119643 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cd693-97db-48b2-a564-2bb58f60e038 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:37.120380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cd693-97db-48b2-a564-2bb58f60e038 admin admin] PATH_INFO: `/v3/projects/1b9414cce74d4a08beae049e94d44e3d/users/3c63345e09e346d6b5963d7e009107b6/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:37.143109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cd693-97db-48b2-a564-2bb58f60e038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b9414cce74d4a08beae049e94d44e3d, user_id=3c63345e09e346d6b5963d7e009107b6, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:37.148764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cd693-97db-48b2-a564-2bb58f60e038 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:37.257093 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1324/2648] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:28:37 2022] PUT /identity/v3/projects/1b9414cce74d4a08beae049e94d44e3d/users/3c63345e09e346d6b5963d7e009107b6/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:28:37.267798 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316860d-e2f7-44da-8fec-1d588e951f7f None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:37.268092 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316860d-e2f7-44da-8fec-1d588e951f7f None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:37.268527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316860d-e2f7-44da-8fec-1d588e951f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:37.308412 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0316860d-e2f7-44da-8fec-1d588e951f7f None None] MFA Rules not processed for user `3c63345e09e346d6b5963d7e009107b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:28:37.310734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0316860d-e2f7-44da-8fec-1d588e951f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:37.386172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1325/2649] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:28:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:28:37.391238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cd693-97db-48b2-a564-2bb58f60e038 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:37.423806 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316860d-e2f7-44da-8fec-1d588e951f7f None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:37.427605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0316860d-e2f7-44da-8fec-1d588e951f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:37.473983 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:28:37.476592 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cd693-97db-48b2-a564-2bb58f60e038 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:37.481668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfae8ff-c7f5-4c47-96b9-ea64bd22f05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:37.482338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfae8ff-c7f5-4c47-96b9-ea64bd22f05e admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:37.483146 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfae8ff-c7f5-4c47-96b9-ea64bd22f05e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:37.483612 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfae8ff-c7f5-4c47-96b9-ea64bd22f05e admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:37.484631 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfae8ff-c7f5-4c47-96b9-ea64bd22f05e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:37.486623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfae8ff-c7f5-4c47-96b9-ea64bd22f05e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:37.514078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316860d-e2f7-44da-8fec-1d588e951f7f None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:37.519916 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb2a74-2faf-4a25-9923-fbdddd56d92a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:37.522284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb2a74-2faf-4a25-9923-fbdddd56d92a service placement] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:37.522284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb2a74-2faf-4a25-9923-fbdddd56d92a service placement] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:37.522284 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb2a74-2faf-4a25-9923-fbdddd56d92a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:37.522976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-25fb2a74-2faf-4a25-9923-fbdddd56d92a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:37.541049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb2a74-2faf-4a25-9923-fbdddd56d92a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:37.542790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb2a74-2faf-4a25-9923-fbdddd56d92a service placement] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:37.555943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1326/2650] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:28:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:28:37.611701 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb2a74-2faf-4a25-9923-fbdddd56d92a service placement] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:37.613319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-25fb2a74-2faf-4a25-9923-fbdddd56d92a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:37.631281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb2a74-2faf-4a25-9923-fbdddd56d92a service placement] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:37.635783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb90957-965b-4705-865a-90f218da9894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:37.636526 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb90957-965b-4705-865a-90f218da9894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:37.636931 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb90957-965b-4705-865a-90f218da9894 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:37.637278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1325/2651] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:28:37 2022] POST /identity/v3/projects => generated 375 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:28:37.640573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb90957-965b-4705-865a-90f218da9894 admin admin] PATH_INFO: `/v3/users/ab1a288170374d1a80355c869c78b4ca` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:37.641590 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfae8ff-c7f5-4c47-96b9-ea64bd22f05e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:37.642127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb90957-965b-4705-865a-90f218da9894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab1a288170374d1a80355c869c78b4ca)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:37.644044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb90957-965b-4705-865a-90f218da9894 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:37.711708 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfae8ff-c7f5-4c47-96b9-ea64bd22f05e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:37.716606 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9530af4-1cb2-4e7d-84b1-da04d0de243c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:37.718663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9530af4-1cb2-4e7d-84b1-da04d0de243c admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:37.718663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9530af4-1cb2-4e7d-84b1-da04d0de243c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:37.718663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9530af4-1cb2-4e7d-84b1-da04d0de243c admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:37.719201 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9530af4-1cb2-4e7d-84b1-da04d0de243c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:37.721165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9530af4-1cb2-4e7d-84b1-da04d0de243c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:37.832236 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1326/2652] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:28:37 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:28:37.833671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9530af4-1cb2-4e7d-84b1-da04d0de243c admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:37.844354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9530af4-1cb2-4e7d-84b1-da04d0de243c admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:37.850091 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234f83c-f999-4ead-ab78-575624ed69cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:37.850892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234f83c-f999-4ead-ab78-575624ed69cf admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:37.851041 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bcb90957-965b-4705-865a-90f218da9894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab1a288170374d1a80355c869c78b4ca'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:37.851449 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234f83c-f999-4ead-ab78-575624ed69cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:37.851861 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234f83c-f999-4ead-ab78-575624ed69cf admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:37.852635 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234f83c-f999-4ead-ab78-575624ed69cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:37.854422 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234f83c-f999-4ead-ab78-575624ed69cf admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:37.859263 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bcb90957-965b-4705-865a-90f218da9894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab1a288170374d1a80355c869c78b4ca'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:37.867556 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1327/2653] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:28:37.871760 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234f83c-f999-4ead-ab78-575624ed69cf admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:37.880304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:28:37.883823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234f83c-f999-4ead-ab78-575624ed69cf admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:37.888525 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc40f8-8e89-44ad-b885-4586567ee56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:37.889518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc40f8-8e89-44ad-b885-4586567ee56d admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:37.889940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc40f8-8e89-44ad-b885-4586567ee56d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:37.890222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc40f8-8e89-44ad-b885-4586567ee56d admin admin] PATH_INFO: `/v3/projects/f649d4ab45b349cb817e0f8f7a75c936/users/5d32ca7738184c22ae4cc98643f9de99/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:37.905027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc40f8-8e89-44ad-b885-4586567ee56d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f649d4ab45b349cb817e0f8f7a75c936, user_id=5d32ca7738184c22ae4cc98643f9de99, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:37.907769 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc40f8-8e89-44ad-b885-4586567ee56d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:37.930409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bcb90957-965b-4705-865a-90f218da9894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab1a288170374d1a80355c869c78b4ca'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:37.931100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bcb90957-965b-4705-865a-90f218da9894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab1a288170374d1a80355c869c78b4ca'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:37.945157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bcb90957-965b-4705-865a-90f218da9894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab1a288170374d1a80355c869c78b4ca'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:37.951381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1327/2654] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:28:37 2022] DELETE /identity/v3/users/ab1a288170374d1a80355c869c78b4ca => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:28:38.019023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1328/2655] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:28:37 2022] PUT /identity/v3/projects/f649d4ab45b349cb817e0f8f7a75c936/users/5d32ca7738184c22ae4cc98643f9de99/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:28:38.023110 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb90957-965b-4705-865a-90f218da9894 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:38.025006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-bcb90957-965b-4705-865a-90f218da9894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:38.146438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb90957-965b-4705-865a-90f218da9894 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:38.216462 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:28:38.242928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bc3e1-a057-4807-b781-efc16b152373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:38.244443 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bc3e1-a057-4807-b781-efc16b152373 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:38.245048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bc3e1-a057-4807-b781-efc16b152373 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:38.245503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bc3e1-a057-4807-b781-efc16b152373 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:38.246456 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bc3e1-a057-4807-b781-efc16b152373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:38.249488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bc3e1-a057-4807-b781-efc16b152373 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:38.269247 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1328/2656] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:28:38.272668 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc40f8-8e89-44ad-b885-4586567ee56d admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:38.287397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc40f8-8e89-44ad-b885-4586567ee56d admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:38.303280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebb337-cbab-49fe-ac0a-f7dc34f31585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:38.305617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebb337-cbab-49fe-ac0a-f7dc34f31585 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:38.306623 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebb337-cbab-49fe-ac0a-f7dc34f31585 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:38.306996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebb337-cbab-49fe-ac0a-f7dc34f31585 admin admin] PATH_INFO: `/v3/projects/f649d4ab45b349cb817e0f8f7a75c936/users/5d32ca7738184c22ae4cc98643f9de99/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:38.311319 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebb337-cbab-49fe-ac0a-f7dc34f31585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f649d4ab45b349cb817e0f8f7a75c936, user_id=5d32ca7738184c22ae4cc98643f9de99, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:38.315672 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebb337-cbab-49fe-ac0a-f7dc34f31585 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:38.340587 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1329/2657] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:28:38 2022] PUT /identity/v3/projects/f649d4ab45b349cb817e0f8f7a75c936/users/5d32ca7738184c22ae4cc98643f9de99/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:28:38.370891 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afcfdd-dd42-44a6-8d8a-2c44463518f4 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:38.371305 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afcfdd-dd42-44a6-8d8a-2c44463518f4 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:38.371453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afcfdd-dd42-44a6-8d8a-2c44463518f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:38.406344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-b5afcfdd-dd42-44a6-8d8a-2c44463518f4 None None] MFA Rules not processed for user `5d32ca7738184c22ae4cc98643f9de99`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:28:38.407980 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b5afcfdd-dd42-44a6-8d8a-2c44463518f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:38.461972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebb337-cbab-49fe-ac0a-f7dc34f31585 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:38.463909 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-1bebb337-cbab-49fe-ac0a-f7dc34f31585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:38.508459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1329/2658] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:28:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:28:38.520335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afcfdd-dd42-44a6-8d8a-2c44463518f4 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:38.522796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b5afcfdd-dd42-44a6-8d8a-2c44463518f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:38.565810 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebb337-cbab-49fe-ac0a-f7dc34f31585 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:38.572726 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:38.573380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:38.573669 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:38.573949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] PATH_INFO: `/v3/projects/05b468fa5d3046ebb37fbd43102be471` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:38.576028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b468fa5d3046ebb37fbd43102be471)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:38.578304 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:38.615636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afcfdd-dd42-44a6-8d8a-2c44463518f4 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:38.619612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566444da-688f-4a62-8c09-67b000082417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:38.620277 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566444da-688f-4a62-8c09-67b000082417 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:38.620564 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566444da-688f-4a62-8c09-67b000082417 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:38.621116 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566444da-688f-4a62-8c09-67b000082417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:38.622939 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-566444da-688f-4a62-8c09-67b000082417 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:38.690687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566444da-688f-4a62-8c09-67b000082417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:38.692093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566444da-688f-4a62-8c09-67b000082417 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:38.701637 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b468fa5d3046ebb37fbd43102be471', 'request_id': 'req-bffac189-84e3-4245-8e07-21a1405d09c8'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:38.704960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1330/2659] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:28:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:28:38.709052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b468fa5d3046ebb37fbd43102be471', 'request_id': 'req-bffac189-84e3-4245-8e07-21a1405d09c8'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:38.768617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b468fa5d3046ebb37fbd43102be471', 'request_id': 'req-bffac189-84e3-4245-8e07-21a1405d09c8'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:38.780902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] The token cache is being invalidate because project 05b468fa5d3046ebb37fbd43102be471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:28:38.781199 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:38.781904 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:28:38.783698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1330/2660] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:28:38 2022] DELETE /identity/v3/projects/05b468fa5d3046ebb37fbd43102be471 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:28:45.114999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566444da-688f-4a62-8c09-67b000082417 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:28:45.116389 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-566444da-688f-4a62-8c09-67b000082417 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:45.127836 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566444da-688f-4a62-8c09-67b000082417 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:28:45.140347 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:28:45.175383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332bc13-b44e-4e26-b888-64e3b7c19d59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:28:45.175991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332bc13-b44e-4e26-b888-64e3b7c19d59 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:28:45.176268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332bc13-b44e-4e26-b888-64e3b7c19d59 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:28:45.176537 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332bc13-b44e-4e26-b888-64e3b7c19d59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:28:45.178381 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9332bc13-b44e-4e26-b888-64e3b7c19d59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:28:45.240068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332bc13-b44e-4e26-b888-64e3b7c19d59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:28:45.241519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332bc13-b44e-4e26-b888-64e3b7c19d59 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:28:45.251466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1331/2661] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:28:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:28:48.777649 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=64,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:28:55.246559 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=42 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:12.367405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:12.368992 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:12.440043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffac189-84e3-4245-8e07-21a1405d09c8 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:12.444513 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603f59a-c407-4fff-9b6e-617f1e539d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:12.445163 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603f59a-c407-4fff-9b6e-617f1e539d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:12.445446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603f59a-c407-4fff-9b6e-617f1e539d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:12.445720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603f59a-c407-4fff-9b6e-617f1e539d84 admin admin] PATH_INFO: `/v3/users/3c63345e09e346d6b5963d7e009107b6` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:12.446832 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603f59a-c407-4fff-9b6e-617f1e539d84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c63345e09e346d6b5963d7e009107b6)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:12.448767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603f59a-c407-4fff-9b6e-617f1e539d84 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:12.621005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5603f59a-c407-4fff-9b6e-617f1e539d84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c63345e09e346d6b5963d7e009107b6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:12.681030 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5603f59a-c407-4fff-9b6e-617f1e539d84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c63345e09e346d6b5963d7e009107b6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:12.681697 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5603f59a-c407-4fff-9b6e-617f1e539d84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c63345e09e346d6b5963d7e009107b6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:12.689042 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5603f59a-c407-4fff-9b6e-617f1e539d84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c63345e09e346d6b5963d7e009107b6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:12.700781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-5603f59a-c407-4fff-9b6e-617f1e539d84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c63345e09e346d6b5963d7e009107b6'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:12.706747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1331/2662] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:29:12 2022] DELETE /identity/v3/users/3c63345e09e346d6b5963d7e009107b6 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:12.711404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10217092-46d4-465e-ae19-0b5ce9aec3fb None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:12.711899 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10217092-46d4-465e-ae19-0b5ce9aec3fb None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:12.712004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10217092-46d4-465e-ae19-0b5ce9aec3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:12.731109 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-10217092-46d4-465e-ae19-0b5ce9aec3fb None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:12.732292 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-10217092-46d4-465e-ae19-0b5ce9aec3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:12.788902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1332/2663] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:12.799418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603f59a-c407-4fff-9b6e-617f1e539d84 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:12.801016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5603f59a-c407-4fff-9b6e-617f1e539d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:12.858622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603f59a-c407-4fff-9b6e-617f1e539d84 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:12.862287 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80da39a-82c9-4049-ab10-70a8b42e9872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:12.862876 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80da39a-82c9-4049-ab10-70a8b42e9872 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:12.863180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80da39a-82c9-4049-ab10-70a8b42e9872 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:12.863452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80da39a-82c9-4049-ab10-70a8b42e9872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:12.872245 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80da39a-82c9-4049-ab10-70a8b42e9872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:12.873637 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80da39a-82c9-4049-ab10-70a8b42e9872 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:12.884447 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1332/2664] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:29:13.297123 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10217092-46d4-465e-ae19-0b5ce9aec3fb None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:13.298707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-10217092-46d4-465e-ae19-0b5ce9aec3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:13.310544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10217092-46d4-465e-ae19-0b5ce9aec3fb None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:13.315588 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:13.316231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:13.316511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:13.316809 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] PATH_INFO: `/v3/projects/1b9414cce74d4a08beae049e94d44e3d` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:13.317923 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b9414cce74d4a08beae049e94d44e3d)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:13.319848 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:13.409673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b9414cce74d4a08beae049e94d44e3d', 'request_id': 'req-c058c625-c964-452b-bfff-e4c1f6c68b9e'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:13.415759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b9414cce74d4a08beae049e94d44e3d', 'request_id': 'req-c058c625-c964-452b-bfff-e4c1f6c68b9e'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:13.421799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b9414cce74d4a08beae049e94d44e3d', 'request_id': 'req-c058c625-c964-452b-bfff-e4c1f6c68b9e'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:13.477076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] The token cache is being invalidate because project 1b9414cce74d4a08beae049e94d44e3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:29:13.477371 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:13.477993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:13.479782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1333/2665] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:29:13 2022] DELETE /identity/v3/projects/1b9414cce74d4a08beae049e94d44e3d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:13.483543 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80da39a-82c9-4049-ab10-70a8b42e9872 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:13.493346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80da39a-82c9-4049-ab10-70a8b42e9872 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:13.511333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:13.549374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd0e84-3dda-4d26-a85a-9fd61f013623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:13.558157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd0e84-3dda-4d26-a85a-9fd61f013623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:13.558157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd0e84-3dda-4d26-a85a-9fd61f013623 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:13.558157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd0e84-3dda-4d26-a85a-9fd61f013623 admin admin] PATH_INFO: `/v3/users/5d32ca7738184c22ae4cc98643f9de99` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:13.558157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd0e84-3dda-4d26-a85a-9fd61f013623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d32ca7738184c22ae4cc98643f9de99)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:13.558157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd0e84-3dda-4d26-a85a-9fd61f013623 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:13.701047 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-30bd0e84-3dda-4d26-a85a-9fd61f013623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d32ca7738184c22ae4cc98643f9de99'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:13.767240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-30bd0e84-3dda-4d26-a85a-9fd61f013623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d32ca7738184c22ae4cc98643f9de99'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:13.767852 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-30bd0e84-3dda-4d26-a85a-9fd61f013623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d32ca7738184c22ae4cc98643f9de99'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:13.774991 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-30bd0e84-3dda-4d26-a85a-9fd61f013623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d32ca7738184c22ae4cc98643f9de99'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:13.787074 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-30bd0e84-3dda-4d26-a85a-9fd61f013623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d32ca7738184c22ae4cc98643f9de99'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:13.793002 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1333/2666] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:29:13 2022] DELETE /identity/v3/users/5d32ca7738184c22ae4cc98643f9de99 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:14.185714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:14.185714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:14.198686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058c625-c964-452b-bfff-e4c1f6c68b9e admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:14.257994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe0456-92f3-478e-969f-405a04f7e92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:14.258619 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe0456-92f3-478e-969f-405a04f7e92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:14.258899 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe0456-92f3-478e-969f-405a04f7e92d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:14.259172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe0456-92f3-478e-969f-405a04f7e92d admin admin] PATH_INFO: `/v3/projects/f649d4ab45b349cb817e0f8f7a75c936` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:14.260288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe0456-92f3-478e-969f-405a04f7e92d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f649d4ab45b349cb817e0f8f7a75c936)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:14.262213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe0456-92f3-478e-969f-405a04f7e92d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:14.356962 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:14.373348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8cfe0456-92f3-478e-969f-405a04f7e92d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f649d4ab45b349cb817e0f8f7a75c936', 'request_id': 'req-8cfe0456-92f3-478e-969f-405a04f7e92d'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:14.379970 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8cfe0456-92f3-478e-969f-405a04f7e92d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f649d4ab45b349cb817e0f8f7a75c936', 'request_id': 'req-8cfe0456-92f3-478e-969f-405a04f7e92d'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:14.385665 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8cfe0456-92f3-478e-969f-405a04f7e92d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f649d4ab45b349cb817e0f8f7a75c936', 'request_id': 'req-8cfe0456-92f3-478e-969f-405a04f7e92d'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:14.459376 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8cfe0456-92f3-478e-969f-405a04f7e92d admin admin] The token cache is being invalidate because project f649d4ab45b349cb817e0f8f7a75c936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:29:14.459602 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8cfe0456-92f3-478e-969f-405a04f7e92d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:14.460226 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8cfe0456-92f3-478e-969f-405a04f7e92d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:14.461784 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1334/2667] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:29:14 2022] DELETE /identity/v3/projects/f649d4ab45b349cb817e0f8f7a75c936 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:14.466593 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4591fca-f94b-4cac-83b7-ec4638455a9a None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:14.466907 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4591fca-f94b-4cac-83b7-ec4638455a9a None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:14.467183 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4591fca-f94b-4cac-83b7-ec4638455a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:14.487580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-b4591fca-f94b-4cac-83b7-ec4638455a9a None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:14.488811 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b4591fca-f94b-4cac-83b7-ec4638455a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:14.549799 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1334/2668] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:14.560831 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da706a86-7357-4ba3-904b-ec9da5afde6d None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:14.561427 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da706a86-7357-4ba3-904b-ec9da5afde6d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:14.561890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da706a86-7357-4ba3-904b-ec9da5afde6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:14.582307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-da706a86-7357-4ba3-904b-ec9da5afde6d None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:14.583485 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-da706a86-7357-4ba3-904b-ec9da5afde6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:14.591134 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1335/2669] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:14.596507 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4591fca-f94b-4cac-83b7-ec4638455a9a None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:14.606072 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4591fca-f94b-4cac-83b7-ec4638455a9a None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:14.610840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a81116-3346-484f-8eef-ff44733f1ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:14.611423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81116-3346-484f-8eef-ff44733f1ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:14.611701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81116-3346-484f-8eef-ff44733f1ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:14.611973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81116-3346-484f-8eef-ff44733f1ba8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:14.612609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a81116-3346-484f-8eef-ff44733f1ba8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:29:14.612943 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a81116-3346-484f-8eef-ff44733f1ba8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:14.614541 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a81116-3346-484f-8eef-ff44733f1ba8 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:14.625654 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1335/2670] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:29:14.630074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da706a86-7357-4ba3-904b-ec9da5afde6d None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:14.639159 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da706a86-7357-4ba3-904b-ec9da5afde6d None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:14.644621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751613f5-bbbf-4921-9673-86b739a8ac1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:14.645238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751613f5-bbbf-4921-9673-86b739a8ac1e admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:14.645518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751613f5-bbbf-4921-9673-86b739a8ac1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:14.645817 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751613f5-bbbf-4921-9673-86b739a8ac1e admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:14.646445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751613f5-bbbf-4921-9673-86b739a8ac1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:14.648320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751613f5-bbbf-4921-9673-86b739a8ac1e admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:14.736354 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1336/2671] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:29:14 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:29:14.740381 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a81116-3346-484f-8eef-ff44733f1ba8 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:14.750481 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a81116-3346-484f-8eef-ff44733f1ba8 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:14.776198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:14.806806 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e96244-5257-4160-b715-cb8c6844f878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:14.807394 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e96244-5257-4160-b715-cb8c6844f878 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:14.807671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e96244-5257-4160-b715-cb8c6844f878 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:14.807989 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e96244-5257-4160-b715-cb8c6844f878 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:14.808950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e96244-5257-4160-b715-cb8c6844f878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:14.811178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e96244-5257-4160-b715-cb8c6844f878 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:14.929175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1336/2672] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:29:14 2022] POST /identity/v3/users => generated 294 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:29:14.933245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751613f5-bbbf-4921-9673-86b739a8ac1e admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:14.944032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751613f5-bbbf-4921-9673-86b739a8ac1e admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:14.948717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c678bd-4e4e-4455-83a2-2ac58d98b826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:14.950850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c678bd-4e4e-4455-83a2-2ac58d98b826 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:14.950850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c678bd-4e4e-4455-83a2-2ac58d98b826 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:14.950850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c678bd-4e4e-4455-83a2-2ac58d98b826 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:14.950850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c678bd-4e4e-4455-83a2-2ac58d98b826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:14.952075 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c678bd-4e4e-4455-83a2-2ac58d98b826 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:14.962803 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1337/2673] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:14.966596 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e96244-5257-4160-b715-cb8c6844f878 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:14.976428 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e96244-5257-4160-b715-cb8c6844f878 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:14.981049 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949af36-8d6a-462f-bdad-8ef1aa18de7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:14.981641 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949af36-8d6a-462f-bdad-8ef1aa18de7f admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:14.981923 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949af36-8d6a-462f-bdad-8ef1aa18de7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:14.982198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949af36-8d6a-462f-bdad-8ef1aa18de7f admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:14.982830 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949af36-8d6a-462f-bdad-8ef1aa18de7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:14.984553 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949af36-8d6a-462f-bdad-8ef1aa18de7f admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:14.994358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1337/2674] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:14.998161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c678bd-4e4e-4455-83a2-2ac58d98b826 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:15.008318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c678bd-4e4e-4455-83a2-2ac58d98b826 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:15.012906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92727fb9-f5e6-4b57-9e96-1299f7ee1f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:15.013543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92727fb9-f5e6-4b57-9e96-1299f7ee1f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:15.013853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92727fb9-f5e6-4b57-9e96-1299f7ee1f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:15.014126 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92727fb9-f5e6-4b57-9e96-1299f7ee1f12 admin admin] PATH_INFO: `/v3/projects/79137b8b0f7345499a7797b6a6da8541/users/86fb20da7d8c434cb3fbe206780c72ee/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:15.028958 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92727fb9-f5e6-4b57-9e96-1299f7ee1f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79137b8b0f7345499a7797b6a6da8541, user_id=86fb20da7d8c434cb3fbe206780c72ee, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:15.031721 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92727fb9-f5e6-4b57-9e96-1299f7ee1f12 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:15.094746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1338/2675] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:29:14 2022] PUT /identity/v3/projects/79137b8b0f7345499a7797b6a6da8541/users/86fb20da7d8c434cb3fbe206780c72ee/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:29:15.104996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5eb7bb-4dd1-4e7a-a8a9-fdad4a81601c None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:15.105285 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5eb7bb-4dd1-4e7a-a8a9-fdad4a81601c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:15.105559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5eb7bb-4dd1-4e7a-a8a9-fdad4a81601c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:15.133156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-5f5eb7bb-4dd1-4e7a-a8a9-fdad4a81601c None None] MFA Rules not processed for user `86fb20da7d8c434cb3fbe206780c72ee`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:15.134356 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5f5eb7bb-4dd1-4e7a-a8a9-fdad4a81601c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:15.197318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1338/2676] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:15.201792 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92727fb9-f5e6-4b57-9e96-1299f7ee1f12 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:15.263999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92727fb9-f5e6-4b57-9e96-1299f7ee1f12 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:15.269485 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f8688-acbd-417c-a32b-cb6678cb0172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:15.270083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f8688-acbd-417c-a32b-cb6678cb0172 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:15.270359 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f8688-acbd-417c-a32b-cb6678cb0172 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:15.270656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f8688-acbd-417c-a32b-cb6678cb0172 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:15.271268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f8688-acbd-417c-a32b-cb6678cb0172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:15.273168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f8688-acbd-417c-a32b-cb6678cb0172 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:15.351095 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1339/2677] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:29:15 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:29:15.355168 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5eb7bb-4dd1-4e7a-a8a9-fdad4a81601c None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:15.366024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5eb7bb-4dd1-4e7a-a8a9-fdad4a81601c None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:15.421827 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fb1cd-b241-4b9b-898d-f0ed4ccd9f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:15.422509 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fb1cd-b241-4b9b-898d-f0ed4ccd9f1b admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:15.422790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fb1cd-b241-4b9b-898d-f0ed4ccd9f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:15.423088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fb1cd-b241-4b9b-898d-f0ed4ccd9f1b admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:15.423710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fb1cd-b241-4b9b-898d-f0ed4ccd9f1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:15.425670 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fb1cd-b241-4b9b-898d-f0ed4ccd9f1b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:15.511531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1339/2678] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:29:15 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:29:15.516232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f8688-acbd-417c-a32b-cb6678cb0172 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:15.526747 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f8688-acbd-417c-a32b-cb6678cb0172 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:15.532185 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8af899-8640-488e-8d9f-4c99ce66278d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:15.532818 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8af899-8640-488e-8d9f-4c99ce66278d admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:15.538507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8af899-8640-488e-8d9f-4c99ce66278d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:15.538507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8af899-8640-488e-8d9f-4c99ce66278d admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:15.538507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8af899-8640-488e-8d9f-4c99ce66278d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:15.538507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8af899-8640-488e-8d9f-4c99ce66278d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:15.546503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1340/2679] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:15.550747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fb1cd-b241-4b9b-898d-f0ed4ccd9f1b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:15.560797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fb1cd-b241-4b9b-898d-f0ed4ccd9f1b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:15.565593 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486f53b-6a66-4f73-aad9-9f3d37867321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:15.566186 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486f53b-6a66-4f73-aad9-9f3d37867321 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:15.566467 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486f53b-6a66-4f73-aad9-9f3d37867321 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:15.566741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486f53b-6a66-4f73-aad9-9f3d37867321 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:15.567399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486f53b-6a66-4f73-aad9-9f3d37867321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:15.569031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486f53b-6a66-4f73-aad9-9f3d37867321 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:15.582961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1340/2680] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:15.584054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8af899-8640-488e-8d9f-4c99ce66278d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:15.595073 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8af899-8640-488e-8d9f-4c99ce66278d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:15.601873 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ddfff-3c65-4d46-9600-2033fdad7e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:15.602557 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ddfff-3c65-4d46-9600-2033fdad7e11 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:15.602832 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ddfff-3c65-4d46-9600-2033fdad7e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:15.603102 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ddfff-3c65-4d46-9600-2033fdad7e11 admin admin] PATH_INFO: `/v3/projects/22273fe21f4c4ca3939a85b1f2a3d2f7/users/41cb8854fe4740a0a91994b090daf980/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:15.622140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ddfff-3c65-4d46-9600-2033fdad7e11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22273fe21f4c4ca3939a85b1f2a3d2f7, user_id=41cb8854fe4740a0a91994b090daf980, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:15.625560 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ddfff-3c65-4d46-9600-2033fdad7e11 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:15.696223 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:15.700839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1341/2681] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:29:15 2022] PUT /identity/v3/projects/22273fe21f4c4ca3939a85b1f2a3d2f7/users/41cb8854fe4740a0a91994b090daf980/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:15.713282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415cf32-9ac6-4db0-9e5d-5ae38caa2df1 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:15.713589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415cf32-9ac6-4db0-9e5d-5ae38caa2df1 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:15.713911 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415cf32-9ac6-4db0-9e5d-5ae38caa2df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:15.744514 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e415cf32-9ac6-4db0-9e5d-5ae38caa2df1 None None] MFA Rules not processed for user `41cb8854fe4740a0a91994b090daf980`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:15.745735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e415cf32-9ac6-4db0-9e5d-5ae38caa2df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:15.750143 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:15.809011 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1341/2682] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:15.813594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ddfff-3c65-4d46-9600-2033fdad7e11 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:15.873853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ddfff-3c65-4d46-9600-2033fdad7e11 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:15.878544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c8d51-daef-4849-96fb-665ec8a434dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:15.879135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c8d51-daef-4849-96fb-665ec8a434dc admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:15.879418 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c8d51-daef-4849-96fb-665ec8a434dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:15.879694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c8d51-daef-4849-96fb-665ec8a434dc admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:15.880317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c8d51-daef-4849-96fb-665ec8a434dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:15.882292 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c8d51-daef-4849-96fb-665ec8a434dc admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:15.976600 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1342/2683] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:29:15 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:29:15.980666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415cf32-9ac6-4db0-9e5d-5ae38caa2df1 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:15.992406 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415cf32-9ac6-4db0-9e5d-5ae38caa2df1 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:16.048378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36352fc1-d797-4693-a258-c5e4763d834b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:16.048996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36352fc1-d797-4693-a258-c5e4763d834b admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:16.049276 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36352fc1-d797-4693-a258-c5e4763d834b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:16.049549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36352fc1-d797-4693-a258-c5e4763d834b admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:16.050157 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36352fc1-d797-4693-a258-c5e4763d834b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:16.052054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36352fc1-d797-4693-a258-c5e4763d834b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:16.134774 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1342/2684] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:29:15 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:29:16.138646 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c8d51-daef-4849-96fb-665ec8a434dc admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:16.149124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c8d51-daef-4849-96fb-665ec8a434dc admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:16.153714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca979737-9c14-41a1-a3b4-32ef9646a808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:16.154306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca979737-9c14-41a1-a3b4-32ef9646a808 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:16.154587 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca979737-9c14-41a1-a3b4-32ef9646a808 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:16.154860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca979737-9c14-41a1-a3b4-32ef9646a808 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:16.155498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca979737-9c14-41a1-a3b4-32ef9646a808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:16.157240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca979737-9c14-41a1-a3b4-32ef9646a808 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:16.167527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1343/2685] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:16.172433 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36352fc1-d797-4693-a258-c5e4763d834b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:16.182285 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36352fc1-d797-4693-a258-c5e4763d834b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:16.187446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447197b-323a-454b-a7bf-960277439d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:16.188114 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447197b-323a-454b-a7bf-960277439d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:16.188393 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447197b-323a-454b-a7bf-960277439d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:16.188693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447197b-323a-454b-a7bf-960277439d8b admin admin] PATH_INFO: `/v3/projects/7d11213f2f41406e8057c18e1e6c41a0/users/dd73d3642424426eb629fd554dca5031/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:16.203472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447197b-323a-454b-a7bf-960277439d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d11213f2f41406e8057c18e1e6c41a0, user_id=dd73d3642424426eb629fd554dca5031, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:16.206265 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447197b-323a-454b-a7bf-960277439d8b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:16.304358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1343/2686] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:29:16 2022] PUT /identity/v3/projects/7d11213f2f41406e8057c18e1e6c41a0/users/dd73d3642424426eb629fd554dca5031/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:16.308475 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca979737-9c14-41a1-a3b4-32ef9646a808 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:16.318558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca979737-9c14-41a1-a3b4-32ef9646a808 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:16.374602 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4c827-bf7c-4313-b70a-dda4b6692803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:16.375257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4c827-bf7c-4313-b70a-dda4b6692803 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:16.375537 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4c827-bf7c-4313-b70a-dda4b6692803 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:16.375808 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4c827-bf7c-4313-b70a-dda4b6692803 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:16.376445 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4c827-bf7c-4313-b70a-dda4b6692803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:16.378067 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4c827-bf7c-4313-b70a-dda4b6692803 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:16.387636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1344/2687] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:16.392277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447197b-323a-454b-a7bf-960277439d8b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:16.402523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447197b-323a-454b-a7bf-960277439d8b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:16.407841 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615626f-1f10-448d-a117-cf1c91e364c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:16.408478 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615626f-1f10-448d-a117-cf1c91e364c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:16.408794 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615626f-1f10-448d-a117-cf1c91e364c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:16.409072 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615626f-1f10-448d-a117-cf1c91e364c6 admin admin] PATH_INFO: `/v3/projects/7d11213f2f41406e8057c18e1e6c41a0/users/dd73d3642424426eb629fd554dca5031/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:16.411005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615626f-1f10-448d-a117-cf1c91e364c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d11213f2f41406e8057c18e1e6c41a0, user_id=dd73d3642424426eb629fd554dca5031, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:16.413680 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615626f-1f10-448d-a117-cf1c91e364c6 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:16.424176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1344/2688] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:29:16 2022] PUT /identity/v3/projects/7d11213f2f41406e8057c18e1e6c41a0/users/dd73d3642424426eb629fd554dca5031/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:29:16.434031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1b3f1-a5d3-4384-94cf-d2f9e4620e87 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:16.434319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1b3f1-a5d3-4384-94cf-d2f9e4620e87 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:16.434591 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1b3f1-a5d3-4384-94cf-d2f9e4620e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:16.462312 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-1ac1b3f1-a5d3-4384-94cf-d2f9e4620e87 None None] MFA Rules not processed for user `dd73d3642424426eb629fd554dca5031`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:16.463492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-1ac1b3f1-a5d3-4384-94cf-d2f9e4620e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:16.521234 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:16.529663 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1345/2689] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:16.540509 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615626f-1f10-448d-a117-cf1c91e364c6 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:16.542275 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-8615626f-1f10-448d-a117-cf1c91e364c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:16.599765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615626f-1f10-448d-a117-cf1c91e364c6 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:16.603405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b629ce-afb6-4d7c-9851-0e22a7ad7287 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:16.604000 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b629ce-afb6-4d7c-9851-0e22a7ad7287 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:16.604307 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b629ce-afb6-4d7c-9851-0e22a7ad7287 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:16.604578 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b629ce-afb6-4d7c-9851-0e22a7ad7287 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:16.613909 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b629ce-afb6-4d7c-9851-0e22a7ad7287 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:16.615312 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b629ce-afb6-4d7c-9851-0e22a7ad7287 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:16.620105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:16.625443 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1345/2690] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:29:17.605186 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1b3f1-a5d3-4384-94cf-d2f9e4620e87 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:17.615219 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1b3f1-a5d3-4384-94cf-d2f9e4620e87 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:17.618754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204bfaa-a630-4c1c-b301-2e0418a74ab8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:17.619345 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204bfaa-a630-4c1c-b301-2e0418a74ab8 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:17.619623 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204bfaa-a630-4c1c-b301-2e0418a74ab8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:17.619896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204bfaa-a630-4c1c-b301-2e0418a74ab8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:17.676587 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204bfaa-a630-4c1c-b301-2e0418a74ab8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:17.678038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204bfaa-a630-4c1c-b301-2e0418a74ab8 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:17.687504 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1346/2691] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:29:22.554095 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b629ce-afb6-4d7c-9851-0e22a7ad7287 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:22.563387 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b629ce-afb6-4d7c-9851-0e22a7ad7287 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:22.567008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63873d2b-d280-4f5a-9172-36a3574a1b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:22.567588 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63873d2b-d280-4f5a-9172-36a3574a1b30 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:22.567870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63873d2b-d280-4f5a-9172-36a3574a1b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:22.568138 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63873d2b-d280-4f5a-9172-36a3574a1b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:22.623723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63873d2b-d280-4f5a-9172-36a3574a1b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:22.625742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63873d2b-d280-4f5a-9172-36a3574a1b30 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:22.634555 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1346/2692] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:29:26.206358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204bfaa-a630-4c1c-b301-2e0418a74ab8 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:26.208090 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f204bfaa-a630-4c1c-b301-2e0418a74ab8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:26.220192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204bfaa-a630-4c1c-b301-2e0418a74ab8 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:26.223901 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a9e78-3a75-4bf8-a5ea-7396fa6167c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:26.224493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a9e78-3a75-4bf8-a5ea-7396fa6167c0 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:26.224810 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a9e78-3a75-4bf8-a5ea-7396fa6167c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:26.225110 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a9e78-3a75-4bf8-a5ea-7396fa6167c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:26.226784 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3b7a9e78-3a75-4bf8-a5ea-7396fa6167c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:26.299688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a9e78-3a75-4bf8-a5ea-7396fa6167c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:26.301194 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a9e78-3a75-4bf8-a5ea-7396fa6167c0 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:26.312181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1347/2693] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:29:26.376833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63873d2b-d280-4f5a-9172-36a3574a1b30 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:26.439102 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63873d2b-d280-4f5a-9172-36a3574a1b30 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:26.443695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82782851-9dc0-4aec-b2e8-1f804434bc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:26.444260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82782851-9dc0-4aec-b2e8-1f804434bc74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:26.444561 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82782851-9dc0-4aec-b2e8-1f804434bc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:26.444850 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82782851-9dc0-4aec-b2e8-1f804434bc74 admin admin] PATH_INFO: `/v3/users/86fb20da7d8c434cb3fbe206780c72ee` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:26.445955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82782851-9dc0-4aec-b2e8-1f804434bc74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86fb20da7d8c434cb3fbe206780c72ee)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:26.447873 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82782851-9dc0-4aec-b2e8-1f804434bc74 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:26.616110 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-82782851-9dc0-4aec-b2e8-1f804434bc74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86fb20da7d8c434cb3fbe206780c72ee'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:26.688455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-82782851-9dc0-4aec-b2e8-1f804434bc74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86fb20da7d8c434cb3fbe206780c72ee'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:26.689302 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-82782851-9dc0-4aec-b2e8-1f804434bc74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86fb20da7d8c434cb3fbe206780c72ee'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:26.696499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-82782851-9dc0-4aec-b2e8-1f804434bc74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86fb20da7d8c434cb3fbe206780c72ee'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:26.708874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-82782851-9dc0-4aec-b2e8-1f804434bc74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86fb20da7d8c434cb3fbe206780c72ee'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:26.714737 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1347/2694] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:29:26 2022] DELETE /identity/v3/users/86fb20da7d8c434cb3fbe206780c72ee => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:26.719796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f44ff0-dafa-4628-9b0b-db72685b77e8 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:26.720082 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f44ff0-dafa-4628-9b0b-db72685b77e8 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:26.720394 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f44ff0-dafa-4628-9b0b-db72685b77e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:26.738767 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a5f44ff0-dafa-4628-9b0b-db72685b77e8 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:26.739948 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a5f44ff0-dafa-4628-9b0b-db72685b77e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:26.796658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1348/2695] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:26.806482 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82782851-9dc0-4aec-b2e8-1f804434bc74 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:26.807859 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-82782851-9dc0-4aec-b2e8-1f804434bc74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:26.829079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:26.865418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82782851-9dc0-4aec-b2e8-1f804434bc74 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:26.869075 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a487119-dedd-40fa-af66-97c4454ee5ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:26.869667 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a487119-dedd-40fa-af66-97c4454ee5ec service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:26.869946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a487119-dedd-40fa-af66-97c4454ee5ec service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:26.870224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a487119-dedd-40fa-af66-97c4454ee5ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:26.879555 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a487119-dedd-40fa-af66-97c4454ee5ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:26.881019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a487119-dedd-40fa-af66-97c4454ee5ec service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:26.890104 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1348/2696] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:29:27.278799 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f44ff0-dafa-4628-9b0b-db72685b77e8 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:27.280255 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a5f44ff0-dafa-4628-9b0b-db72685b77e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:27.292535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f44ff0-dafa-4628-9b0b-db72685b77e8 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:27.298912 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:27.299410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:27.299689 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:27.299960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] PATH_INFO: `/v3/projects/79137b8b0f7345499a7797b6a6da8541` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:27.301127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79137b8b0f7345499a7797b6a6da8541)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:27.303076 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:27.320292 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:27.383391 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a487119-dedd-40fa-af66-97c4454ee5ec service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:27.385732 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-5a487119-dedd-40fa-af66-97c4454ee5ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:27.393713 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79137b8b0f7345499a7797b6a6da8541', 'request_id': 'req-3560f947-1d92-4fda-8e57-c27efaa45870'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:27.400130 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79137b8b0f7345499a7797b6a6da8541', 'request_id': 'req-3560f947-1d92-4fda-8e57-c27efaa45870'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:27.406353 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79137b8b0f7345499a7797b6a6da8541', 'request_id': 'req-3560f947-1d92-4fda-8e57-c27efaa45870'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:27.443598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a487119-dedd-40fa-af66-97c4454ee5ec service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:27.459605 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] The token cache is being invalidate because project 79137b8b0f7345499a7797b6a6da8541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:29:27.460012 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:27.460787 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:27.462294 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1349/2697] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:29:27 2022] DELETE /identity/v3/projects/79137b8b0f7345499a7797b6a6da8541 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:27.466321 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:27.468060 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:27.492597 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4826e2c-401d-4d13-abb5-eb9c3f22d0e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:27.493229 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4826e2c-401d-4d13-abb5-eb9c3f22d0e1 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:27.493513 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4826e2c-401d-4d13-abb5-eb9c3f22d0e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:27.493903 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4826e2c-401d-4d13-abb5-eb9c3f22d0e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:27.495633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f4826e2c-401d-4d13-abb5-eb9c3f22d0e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:27.541533 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560f947-1d92-4fda-8e57-c27efaa45870 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:27.546395 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07601f5f-3aba-4938-90ea-83070e5b8aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:27.547533 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07601f5f-3aba-4938-90ea-83070e5b8aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:27.548411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07601f5f-3aba-4938-90ea-83070e5b8aff admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:27.548615 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07601f5f-3aba-4938-90ea-83070e5b8aff admin admin] PATH_INFO: `/v3/users/41cb8854fe4740a0a91994b090daf980` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:27.549784 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07601f5f-3aba-4938-90ea-83070e5b8aff admin admin] RBAC: Authorizing `identity:delete_user(user_id=41cb8854fe4740a0a91994b090daf980)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:27.551672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07601f5f-3aba-4938-90ea-83070e5b8aff admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:27.569954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4826e2c-401d-4d13-abb5-eb9c3f22d0e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:27.571399 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4826e2c-401d-4d13-abb5-eb9c3f22d0e1 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:27.582800 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1349/2698] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:29:27.716638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-07601f5f-3aba-4938-90ea-83070e5b8aff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41cb8854fe4740a0a91994b090daf980'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:27.723898 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-07601f5f-3aba-4938-90ea-83070e5b8aff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41cb8854fe4740a0a91994b090daf980'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:27.789419 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-07601f5f-3aba-4938-90ea-83070e5b8aff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41cb8854fe4740a0a91994b090daf980'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:27.790034 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-07601f5f-3aba-4938-90ea-83070e5b8aff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41cb8854fe4740a0a91994b090daf980'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:27.803048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-07601f5f-3aba-4938-90ea-83070e5b8aff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41cb8854fe4740a0a91994b090daf980'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:27.808801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1350/2699] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:29:27 2022] DELETE /identity/v3/users/41cb8854fe4740a0a91994b090daf980 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:28.211092 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4826e2c-401d-4d13-abb5-eb9c3f22d0e1 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:28.225053 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4826e2c-401d-4d13-abb5-eb9c3f22d0e1 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:28.247401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:28.298674 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:28.299341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:28.306488 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:28.306488 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] PATH_INFO: `/v3/projects/22273fe21f4c4ca3939a85b1f2a3d2f7` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:28.306488 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=22273fe21f4c4ca3939a85b1f2a3d2f7)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:28.306488 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:28.403202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22273fe21f4c4ca3939a85b1f2a3d2f7', 'request_id': 'req-4f48148f-df05-4d81-bbfe-a62e6c279e4d'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:28.410306 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22273fe21f4c4ca3939a85b1f2a3d2f7', 'request_id': 'req-4f48148f-df05-4d81-bbfe-a62e6c279e4d'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:28.492533 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22273fe21f4c4ca3939a85b1f2a3d2f7', 'request_id': 'req-4f48148f-df05-4d81-bbfe-a62e6c279e4d'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:28.505290 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] The token cache is being invalidate because project 22273fe21f4c4ca3939a85b1f2a3d2f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:29:28.505579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:28.506247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:28.507723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1350/2700] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:29:28 2022] DELETE /identity/v3/projects/22273fe21f4c4ca3939a85b1f2a3d2f7 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:28.512169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07601f5f-3aba-4938-90ea-83070e5b8aff admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:28.513896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-07601f5f-3aba-4938-90ea-83070e5b8aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:28.532197 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07601f5f-3aba-4938-90ea-83070e5b8aff admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:28.551867 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:28.596283 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2db2f-042f-45b0-9077-7562c0d6e8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:28.597015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2db2f-042f-45b0-9077-7562c0d6e8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:28.597295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2db2f-042f-45b0-9077-7562c0d6e8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:28.597482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2db2f-042f-45b0-9077-7562c0d6e8d2 admin admin] PATH_INFO: `/v3/users/dd73d3642424426eb629fd554dca5031` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:28.598915 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2db2f-042f-45b0-9077-7562c0d6e8d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd73d3642424426eb629fd554dca5031)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:28.601006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2db2f-042f-45b0-9077-7562c0d6e8d2 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:28.812455 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2dd2db2f-042f-45b0-9077-7562c0d6e8d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd73d3642424426eb629fd554dca5031'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:28.819693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2dd2db2f-042f-45b0-9077-7562c0d6e8d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd73d3642424426eb629fd554dca5031'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:28.883383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2dd2db2f-042f-45b0-9077-7562c0d6e8d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd73d3642424426eb629fd554dca5031'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:28.884008 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2dd2db2f-042f-45b0-9077-7562c0d6e8d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd73d3642424426eb629fd554dca5031'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:28.899341 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-2dd2db2f-042f-45b0-9077-7562c0d6e8d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd73d3642424426eb629fd554dca5031'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:28.904361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1351/2701] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:29:28 2022] DELETE /identity/v3/users/dd73d3642424426eb629fd554dca5031 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:29.353275 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:29.354894 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:29.371767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48148f-df05-4d81-bbfe-a62e6c279e4d admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:29.429738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845f508-355f-40f9-a641-fdfd3f49efe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:29.430405 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845f508-355f-40f9-a641-fdfd3f49efe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:29.430626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845f508-355f-40f9-a641-fdfd3f49efe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:29.430896 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845f508-355f-40f9-a641-fdfd3f49efe3 admin admin] PATH_INFO: `/v3/projects/7d11213f2f41406e8057c18e1e6c41a0` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:29.431994 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845f508-355f-40f9-a641-fdfd3f49efe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d11213f2f41406e8057c18e1e6c41a0)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:29.433954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845f508-355f-40f9-a641-fdfd3f49efe3 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:29.552969 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f845f508-355f-40f9-a641-fdfd3f49efe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d11213f2f41406e8057c18e1e6c41a0', 'request_id': 'req-f845f508-355f-40f9-a641-fdfd3f49efe3'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:29.554065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:29.559783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f845f508-355f-40f9-a641-fdfd3f49efe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d11213f2f41406e8057c18e1e6c41a0', 'request_id': 'req-f845f508-355f-40f9-a641-fdfd3f49efe3'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:29.648790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f845f508-355f-40f9-a641-fdfd3f49efe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d11213f2f41406e8057c18e1e6c41a0', 'request_id': 'req-f845f508-355f-40f9-a641-fdfd3f49efe3'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:29.659699 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f845f508-355f-40f9-a641-fdfd3f49efe3 admin admin] The token cache is being invalidate because project 7d11213f2f41406e8057c18e1e6c41a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:29:29.659985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f845f508-355f-40f9-a641-fdfd3f49efe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:29.660608 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f845f508-355f-40f9-a641-fdfd3f49efe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:29.662113 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1351/2702] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:29:29 2022] DELETE /identity/v3/projects/7d11213f2f41406e8057c18e1e6c41a0 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:29.668352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5403c-978c-45a6-b2f9-5113082fed28 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:29.668853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5403c-978c-45a6-b2f9-5113082fed28 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:29.668998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5403c-978c-45a6-b2f9-5113082fed28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:29.689369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-31b5403c-978c-45a6-b2f9-5113082fed28 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:29.690500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-31b5403c-978c-45a6-b2f9-5113082fed28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:29.749234 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1352/2703] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:29.762118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607a626-ac3a-40ca-914a-8aecb8631eea None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:29.762437 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607a626-ac3a-40ca-914a-8aecb8631eea None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:29.762716 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607a626-ac3a-40ca-914a-8aecb8631eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:29.782418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e607a626-ac3a-40ca-914a-8aecb8631eea None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:29.783613 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e607a626-ac3a-40ca-914a-8aecb8631eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:29.791521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1352/2704] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:29.795988 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5403c-978c-45a6-b2f9-5113082fed28 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:29.806331 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5403c-978c-45a6-b2f9-5113082fed28 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:29.810794 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90ece1-f23f-475d-a8fc-681d9c48da97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:29.811379 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90ece1-f23f-475d-a8fc-681d9c48da97 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:29.811659 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90ece1-f23f-475d-a8fc-681d9c48da97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:29.811928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90ece1-f23f-475d-a8fc-681d9c48da97 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:29.813270 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90ece1-f23f-475d-a8fc-681d9c48da97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:29:29.813828 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90ece1-f23f-475d-a8fc-681d9c48da97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:29.815443 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90ece1-f23f-475d-a8fc-681d9c48da97 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:29.826552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1353/2705] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:29:29.830929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607a626-ac3a-40ca-914a-8aecb8631eea None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:29.840021 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607a626-ac3a-40ca-914a-8aecb8631eea None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:29.844891 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46c5a5-dcc2-4372-a870-69d79ad17560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:29.845491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46c5a5-dcc2-4372-a870-69d79ad17560 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:29.845775 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46c5a5-dcc2-4372-a870-69d79ad17560 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:29.846051 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46c5a5-dcc2-4372-a870-69d79ad17560 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:29.846685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46c5a5-dcc2-4372-a870-69d79ad17560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:29.848574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46c5a5-dcc2-4372-a870-69d79ad17560 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:29.941506 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1353/2706] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:29:29 2022] POST /identity/v3/projects => generated 399 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:29:29.945760 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90ece1-f23f-475d-a8fc-681d9c48da97 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:29.955819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90ece1-f23f-475d-a8fc-681d9c48da97 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:29.981251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:30.011800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780283c5-a017-49fc-bd40-d877d50cb977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:30.012401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780283c5-a017-49fc-bd40-d877d50cb977 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:30.012718 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780283c5-a017-49fc-bd40-d877d50cb977 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:30.013000 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780283c5-a017-49fc-bd40-d877d50cb977 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:30.013633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780283c5-a017-49fc-bd40-d877d50cb977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:30.015647 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780283c5-a017-49fc-bd40-d877d50cb977 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:30.140682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1354/2707] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:29:29 2022] POST /identity/v3/users => generated 300 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:29:30.145040 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46c5a5-dcc2-4372-a870-69d79ad17560 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:30.155222 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46c5a5-dcc2-4372-a870-69d79ad17560 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:30.160763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d1921-191f-42cc-b22a-969bc7a48dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:30.161354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d1921-191f-42cc-b22a-969bc7a48dbd admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:30.161630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d1921-191f-42cc-b22a-969bc7a48dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:30.161901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d1921-191f-42cc-b22a-969bc7a48dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:30.162536 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d1921-191f-42cc-b22a-969bc7a48dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:30.164118 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d1921-191f-42cc-b22a-969bc7a48dbd admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:30.175473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1354/2708] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:30.179432 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780283c5-a017-49fc-bd40-d877d50cb977 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:30.189676 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780283c5-a017-49fc-bd40-d877d50cb977 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:30.194319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a8118-7517-4716-b2f3-a32798f21fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:30.194855 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a8118-7517-4716-b2f3-a32798f21fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:30.195136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a8118-7517-4716-b2f3-a32798f21fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:30.195411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a8118-7517-4716-b2f3-a32798f21fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:30.196049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a8118-7517-4716-b2f3-a32798f21fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:30.197673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a8118-7517-4716-b2f3-a32798f21fb7 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:30.207733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1355/2709] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:30.211823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d1921-191f-42cc-b22a-969bc7a48dbd admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:30.222413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d1921-191f-42cc-b22a-969bc7a48dbd admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:30.227019 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492339c0-ef47-41d1-a7d6-4563ca913493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:30.227665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492339c0-ef47-41d1-a7d6-4563ca913493 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:30.227950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492339c0-ef47-41d1-a7d6-4563ca913493 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:30.228231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492339c0-ef47-41d1-a7d6-4563ca913493 admin admin] PATH_INFO: `/v3/projects/32d553814a404c25a3a366ca08eb8b0a/users/9d720bb9dbee40ef956c72dd653a1e8c/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:30.244060 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492339c0-ef47-41d1-a7d6-4563ca913493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32d553814a404c25a3a366ca08eb8b0a, user_id=9d720bb9dbee40ef956c72dd653a1e8c, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:30.246803 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492339c0-ef47-41d1-a7d6-4563ca913493 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:30.351954 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1355/2710] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:29:30 2022] PUT /identity/v3/projects/32d553814a404c25a3a366ca08eb8b0a/users/9d720bb9dbee40ef956c72dd653a1e8c/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:29:30.363871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc9e72-b79d-4153-8cf4-2a3ad0407f33 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:30.364160 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc9e72-b79d-4153-8cf4-2a3ad0407f33 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:30.364436 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc9e72-b79d-4153-8cf4-2a3ad0407f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:30.392185 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-accc9e72-b79d-4153-8cf4-2a3ad0407f33 None None] MFA Rules not processed for user `9d720bb9dbee40ef956c72dd653a1e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:30.393430 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-accc9e72-b79d-4153-8cf4-2a3ad0407f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:30.456406 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1356/2711] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:30 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:30.460946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492339c0-ef47-41d1-a7d6-4563ca913493 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:30.522949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492339c0-ef47-41d1-a7d6-4563ca913493 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:30.527589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524230e-7401-40a3-b4fa-f5cf9513eb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:30.528182 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524230e-7401-40a3-b4fa-f5cf9513eb2c admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:30.528457 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524230e-7401-40a3-b4fa-f5cf9513eb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:30.528750 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524230e-7401-40a3-b4fa-f5cf9513eb2c admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:30.529367 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524230e-7401-40a3-b4fa-f5cf9513eb2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:30.531255 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524230e-7401-40a3-b4fa-f5cf9513eb2c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:30.625525 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1356/2712] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:29:30 2022] POST /identity/v3/projects => generated 399 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:29:30.629705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc9e72-b79d-4153-8cf4-2a3ad0407f33 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:30.640196 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc9e72-b79d-4153-8cf4-2a3ad0407f33 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:30.695647 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f257b7-2ae8-4d4d-9a15-5b8ba45c35a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:30.696297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f257b7-2ae8-4d4d-9a15-5b8ba45c35a6 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:30.696524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f257b7-2ae8-4d4d-9a15-5b8ba45c35a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:30.696840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f257b7-2ae8-4d4d-9a15-5b8ba45c35a6 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:30.697469 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f257b7-2ae8-4d4d-9a15-5b8ba45c35a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:30.699419 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f257b7-2ae8-4d4d-9a15-5b8ba45c35a6 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:30.775798 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1357/2713] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:29:30 2022] POST /identity/v3/users => generated 306 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:29:30.776884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524230e-7401-40a3-b4fa-f5cf9513eb2c admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:30.787710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524230e-7401-40a3-b4fa-f5cf9513eb2c admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:30.792344 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a97bab-68aa-4f9c-aea7-0e37db458ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:30.793231 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a97bab-68aa-4f9c-aea7-0e37db458ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:30.793514 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a97bab-68aa-4f9c-aea7-0e37db458ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:30.793789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a97bab-68aa-4f9c-aea7-0e37db458ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:30.794453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a97bab-68aa-4f9c-aea7-0e37db458ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:30.796033 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a97bab-68aa-4f9c-aea7-0e37db458ca7 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:30.808282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1357/2714] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:30.812390 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f257b7-2ae8-4d4d-9a15-5b8ba45c35a6 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:30.823719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f257b7-2ae8-4d4d-9a15-5b8ba45c35a6 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:30.832563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3371fc0-e60e-484d-b340-46d0955f7471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:30.833265 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3371fc0-e60e-484d-b340-46d0955f7471 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:30.833545 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3371fc0-e60e-484d-b340-46d0955f7471 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:30.833819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3371fc0-e60e-484d-b340-46d0955f7471 admin admin] PATH_INFO: `/v3/projects/e32693e9145546678b0256522d05adc1/users/7fb904b524124216bf79ad8d14de8007/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:30.849636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3371fc0-e60e-484d-b340-46d0955f7471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e32693e9145546678b0256522d05adc1, user_id=7fb904b524124216bf79ad8d14de8007, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:30.852245 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3371fc0-e60e-484d-b340-46d0955f7471 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:30.919999 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1358/2715] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:29:30 2022] PUT /identity/v3/projects/e32693e9145546678b0256522d05adc1/users/7fb904b524124216bf79ad8d14de8007/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:30.924149 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a97bab-68aa-4f9c-aea7-0e37db458ca7 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:30.933940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a97bab-68aa-4f9c-aea7-0e37db458ca7 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:30.978327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:30.990440 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7e215-995f-4b78-a53b-655fc090876c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:30.991042 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7e215-995f-4b78-a53b-655fc090876c admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:30.991328 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7e215-995f-4b78-a53b-655fc090876c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:30.991609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7e215-995f-4b78-a53b-655fc090876c admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:30.992252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7e215-995f-4b78-a53b-655fc090876c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:30.993882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7e215-995f-4b78-a53b-655fc090876c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:31.004035 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1358/2716] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:31.008250 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3371fc0-e60e-484d-b340-46d0955f7471 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:31.013627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:31.018843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3371fc0-e60e-484d-b340-46d0955f7471 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:31.023492 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830930ab-d771-4a47-b37e-a0b7b7825c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:31.024135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830930ab-d771-4a47-b37e-a0b7b7825c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:31.024417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830930ab-d771-4a47-b37e-a0b7b7825c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:31.024727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830930ab-d771-4a47-b37e-a0b7b7825c08 admin admin] PATH_INFO: `/v3/projects/e32693e9145546678b0256522d05adc1/users/7fb904b524124216bf79ad8d14de8007/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:31.026673 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830930ab-d771-4a47-b37e-a0b7b7825c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e32693e9145546678b0256522d05adc1, user_id=7fb904b524124216bf79ad8d14de8007, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:31.029272 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830930ab-d771-4a47-b37e-a0b7b7825c08 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:31.039416 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1359/2717] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:29:31 2022] PUT /identity/v3/projects/e32693e9145546678b0256522d05adc1/users/7fb904b524124216bf79ad8d14de8007/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:31.051009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4ecb3-a26e-4d6c-bcb5-1c15c31523eb None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:31.051341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4ecb3-a26e-4d6c-bcb5-1c15c31523eb None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:31.051657 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4ecb3-a26e-4d6c-bcb5-1c15c31523eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:31.079413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-39e4ecb3-a26e-4d6c-bcb5-1c15c31523eb None None] MFA Rules not processed for user `7fb904b524124216bf79ad8d14de8007`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:31.080619 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-39e4ecb3-a26e-4d6c-bcb5-1c15c31523eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:31.148520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1359/2718] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:31 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:31.160367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830930ab-d771-4a47-b37e-a0b7b7825c08 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:31.162188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-830930ab-d771-4a47-b37e-a0b7b7825c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:31.222364 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830930ab-d771-4a47-b37e-a0b7b7825c08 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:31.226109 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535d931-ff34-4692-a122-2a714927d37c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:31.226704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535d931-ff34-4692-a122-2a714927d37c service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:31.226984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535d931-ff34-4692-a122-2a714927d37c service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:31.227257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535d931-ff34-4692-a122-2a714927d37c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:31.236132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535d931-ff34-4692-a122-2a714927d37c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:31.237578 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535d931-ff34-4692-a122-2a714927d37c service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:31.246681 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1360/2719] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:31 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:29:31.456528 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4ecb3-a26e-4d6c-bcb5-1c15c31523eb None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:31.467035 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4ecb3-a26e-4d6c-bcb5-1c15c31523eb None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:31.470847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13153eb-c04f-4693-853c-de4bca36f800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:31.471440 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13153eb-c04f-4693-853c-de4bca36f800 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:31.471717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13153eb-c04f-4693-853c-de4bca36f800 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:31.471986 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13153eb-c04f-4693-853c-de4bca36f800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:31.530097 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13153eb-c04f-4693-853c-de4bca36f800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:31.531554 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13153eb-c04f-4693-853c-de4bca36f800 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:31.541040 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1360/2720] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:31 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:29:36.560285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535d931-ff34-4692-a122-2a714927d37c service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:36.621728 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535d931-ff34-4692-a122-2a714927d37c service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:36.626764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265220e-0a19-413f-9ba8-af48840d69a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:36.627372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265220e-0a19-413f-9ba8-af48840d69a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:36.627652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265220e-0a19-413f-9ba8-af48840d69a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:36.627977 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265220e-0a19-413f-9ba8-af48840d69a6 admin admin] PATH_INFO: `/v3/users/9d720bb9dbee40ef956c72dd653a1e8c` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:36.629049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265220e-0a19-413f-9ba8-af48840d69a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d720bb9dbee40ef956c72dd653a1e8c)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:36.630966 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265220e-0a19-413f-9ba8-af48840d69a6 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:36.766087 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8265220e-0a19-413f-9ba8-af48840d69a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d720bb9dbee40ef956c72dd653a1e8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:36.772825 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8265220e-0a19-413f-9ba8-af48840d69a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d720bb9dbee40ef956c72dd653a1e8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:36.836857 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8265220e-0a19-413f-9ba8-af48840d69a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d720bb9dbee40ef956c72dd653a1e8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:36.837478 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8265220e-0a19-413f-9ba8-af48840d69a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d720bb9dbee40ef956c72dd653a1e8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:36.850697 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-8265220e-0a19-413f-9ba8-af48840d69a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d720bb9dbee40ef956c72dd653a1e8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:36.856627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1361/2721] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:29:36 2022] DELETE /identity/v3/users/9d720bb9dbee40ef956c72dd653a1e8c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:36.861211 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38423d90-e9fb-4ef1-b3a8-716a47efc071 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:36.861678 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38423d90-e9fb-4ef1-b3a8-716a47efc071 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:36.861789 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38423d90-e9fb-4ef1-b3a8-716a47efc071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:36.881526 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-38423d90-e9fb-4ef1-b3a8-716a47efc071 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:36.882663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-38423d90-e9fb-4ef1-b3a8-716a47efc071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:36.940715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1361/2722] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:36.950417 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265220e-0a19-413f-9ba8-af48840d69a6 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:36.951829 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8265220e-0a19-413f-9ba8-af48840d69a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:36.966202 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:37.010146 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265220e-0a19-413f-9ba8-af48840d69a6 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:37.013794 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ee0f4-2dbd-417f-a205-694547c3d4d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:37.014400 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ee0f4-2dbd-417f-a205-694547c3d4d1 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:37.014678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ee0f4-2dbd-417f-a205-694547c3d4d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:37.014950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ee0f4-2dbd-417f-a205-694547c3d4d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:37.024180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ee0f4-2dbd-417f-a205-694547c3d4d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:37.026133 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ee0f4-2dbd-417f-a205-694547c3d4d1 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:37.036727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1362/2723] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:29:37.453453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38423d90-e9fb-4ef1-b3a8-716a47efc071 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:37.455075 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-38423d90-e9fb-4ef1-b3a8-716a47efc071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:37.466963 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38423d90-e9fb-4ef1-b3a8-716a47efc071 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:37.472508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:37.473747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:37.474032 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:37.474307 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] PATH_INFO: `/v3/projects/32d553814a404c25a3a366ca08eb8b0a` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:37.475442 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32d553814a404c25a3a366ca08eb8b0a)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:37.477937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:37.485350 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=100 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:37.600908 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32d553814a404c25a3a366ca08eb8b0a', 'request_id': 'req-52510546-1534-4e6e-ad90-6abfe00de277'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:37.607151 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32d553814a404c25a3a366ca08eb8b0a', 'request_id': 'req-52510546-1534-4e6e-ad90-6abfe00de277'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:37.670919 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32d553814a404c25a3a366ca08eb8b0a', 'request_id': 'req-52510546-1534-4e6e-ad90-6abfe00de277'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:37.682446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] The token cache is being invalidate because project 32d553814a404c25a3a366ca08eb8b0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:29:37.682734 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:37.683378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:37.684878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1362/2724] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:29:37 2022] DELETE /identity/v3/projects/32d553814a404c25a3a366ca08eb8b0a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:37.689004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ee0f4-2dbd-417f-a205-694547c3d4d1 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:37.703065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ee0f4-2dbd-417f-a205-694547c3d4d1 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:37.754583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e7b5b-a696-45e4-88a0-85e26b615a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:37.755219 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e7b5b-a696-45e4-88a0-85e26b615a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:37.755503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e7b5b-a696-45e4-88a0-85e26b615a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:37.755782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e7b5b-a696-45e4-88a0-85e26b615a78 admin admin] PATH_INFO: `/v3/users/7fb904b524124216bf79ad8d14de8007` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:37.756961 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e7b5b-a696-45e4-88a0-85e26b615a78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb904b524124216bf79ad8d14de8007)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:37.758967 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e7b5b-a696-45e4-88a0-85e26b615a78 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:37.929609 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3a9e7b5b-a696-45e4-88a0-85e26b615a78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb904b524124216bf79ad8d14de8007'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:37.936243 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3a9e7b5b-a696-45e4-88a0-85e26b615a78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb904b524124216bf79ad8d14de8007'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:38.002337 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3a9e7b5b-a696-45e4-88a0-85e26b615a78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb904b524124216bf79ad8d14de8007'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:38.002891 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3a9e7b5b-a696-45e4-88a0-85e26b615a78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb904b524124216bf79ad8d14de8007'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:38.015011 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-3a9e7b5b-a696-45e4-88a0-85e26b615a78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb904b524124216bf79ad8d14de8007'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:38.021671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1363/2725] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:29:37 2022] DELETE /identity/v3/users/7fb904b524124216bf79ad8d14de8007 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:29:38.438672 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:38.440133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:38.454016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52510546-1534-4e6e-ad90-6abfe00de277 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:38.513318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd22ad-0951-4c13-9133-41479daf6621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:38.513871 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd22ad-0951-4c13-9133-41479daf6621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:38.514169 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd22ad-0951-4c13-9133-41479daf6621 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:38.514432 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd22ad-0951-4c13-9133-41479daf6621 admin admin] PATH_INFO: `/v3/projects/e32693e9145546678b0256522d05adc1` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:38.516192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd22ad-0951-4c13-9133-41479daf6621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e32693e9145546678b0256522d05adc1)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:38.518186 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd22ad-0951-4c13-9133-41479daf6621 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:38.609438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-07bd22ad-0951-4c13-9133-41479daf6621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e32693e9145546678b0256522d05adc1', 'request_id': 'req-07bd22ad-0951-4c13-9133-41479daf6621'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:38.615604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-07bd22ad-0951-4c13-9133-41479daf6621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e32693e9145546678b0256522d05adc1', 'request_id': 'req-07bd22ad-0951-4c13-9133-41479daf6621'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:38.676583 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-07bd22ad-0951-4c13-9133-41479daf6621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e32693e9145546678b0256522d05adc1', 'request_id': 'req-07bd22ad-0951-4c13-9133-41479daf6621'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:38.686870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-07bd22ad-0951-4c13-9133-41479daf6621 admin admin] The token cache is being invalidate because project e32693e9145546678b0256522d05adc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:29:38.687162 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-07bd22ad-0951-4c13-9133-41479daf6621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:38.687934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-07bd22ad-0951-4c13-9133-41479daf6621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:38.689627 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1363/2726] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:29:38 2022] DELETE /identity/v3/projects/e32693e9145546678b0256522d05adc1 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:29:38.694499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd69aef-878b-4e9e-9056-e3819d4b5184 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:38.694978 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd69aef-878b-4e9e-9056-e3819d4b5184 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:38.695062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd69aef-878b-4e9e-9056-e3819d4b5184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:38.714875 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-dcd69aef-878b-4e9e-9056-e3819d4b5184 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:38.716123 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-dcd69aef-878b-4e9e-9056-e3819d4b5184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:38.744184 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:38.773689 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1364/2727] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:38.781795 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46607781-9fb1-451e-93c3-ec2caddb770d None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:38.782081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46607781-9fb1-451e-93c3-ec2caddb770d None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:38.782358 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46607781-9fb1-451e-93c3-ec2caddb770d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:38.802449 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-46607781-9fb1-451e-93c3-ec2caddb770d None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:38.803648 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-46607781-9fb1-451e-93c3-ec2caddb770d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:38.811464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1364/2728] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:38.815994 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd69aef-878b-4e9e-9056-e3819d4b5184 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:38.825682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd69aef-878b-4e9e-9056-e3819d4b5184 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:38.831645 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0cc62-fb13-4abe-b643-ec5aa63e1374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:38.832274 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0cc62-fb13-4abe-b643-ec5aa63e1374 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:38.832557 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0cc62-fb13-4abe-b643-ec5aa63e1374 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:38.832849 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0cc62-fb13-4abe-b643-ec5aa63e1374 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:38.833488 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de0cc62-fb13-4abe-b643-ec5aa63e1374 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:29:38.833790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de0cc62-fb13-4abe-b643-ec5aa63e1374 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:38.835372 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de0cc62-fb13-4abe-b643-ec5aa63e1374 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:38.847195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1365/2729] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:29:38.851339 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46607781-9fb1-451e-93c3-ec2caddb770d None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:38.861477 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46607781-9fb1-451e-93c3-ec2caddb770d None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:38.866147 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be94bd3-d906-488f-b058-7d35f59e1b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:38.866740 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be94bd3-d906-488f-b058-7d35f59e1b48 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:38.867022 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be94bd3-d906-488f-b058-7d35f59e1b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:38.867293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be94bd3-d906-488f-b058-7d35f59e1b48 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:38.867916 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be94bd3-d906-488f-b058-7d35f59e1b48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:38.869840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be94bd3-d906-488f-b058-7d35f59e1b48 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:38.884846 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:38.950017 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1365/2730] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:29:38 2022] POST /identity/v3/projects => generated 409 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:29:38.954229 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0cc62-fb13-4abe-b643-ec5aa63e1374 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:38.963998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0cc62-fb13-4abe-b643-ec5aa63e1374 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:39.021175 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5251e-04a5-496a-9c56-617d6fd4b374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:39.021779 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5251e-04a5-496a-9c56-617d6fd4b374 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:39.022062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5251e-04a5-496a-9c56-617d6fd4b374 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:39.022331 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5251e-04a5-496a-9c56-617d6fd4b374 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:39.023022 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5251e-04a5-496a-9c56-617d6fd4b374 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:39.025004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5251e-04a5-496a-9c56-617d6fd4b374 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:39.126681 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1366/2731] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:29:38 2022] POST /identity/v3/users => generated 305 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:29:39.130878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be94bd3-d906-488f-b058-7d35f59e1b48 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:39.141273 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be94bd3-d906-488f-b058-7d35f59e1b48 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:39.146056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab20c3-64ee-4655-801e-3aa597ab72b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:39.146651 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab20c3-64ee-4655-801e-3aa597ab72b4 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:39.146935 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab20c3-64ee-4655-801e-3aa597ab72b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:39.147209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab20c3-64ee-4655-801e-3aa597ab72b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:39.147847 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab20c3-64ee-4655-801e-3aa597ab72b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:39.149503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab20c3-64ee-4655-801e-3aa597ab72b4 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:39.160500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1366/2732] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:39.164159 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5251e-04a5-496a-9c56-617d6fd4b374 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:39.175108 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5251e-04a5-496a-9c56-617d6fd4b374 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:39.179949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3121d-91a9-48f8-b200-d5fae83e598f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:39.180627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3121d-91a9-48f8-b200-d5fae83e598f admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:39.180944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3121d-91a9-48f8-b200-d5fae83e598f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:39.181224 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3121d-91a9-48f8-b200-d5fae83e598f admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:39.181875 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3121d-91a9-48f8-b200-d5fae83e598f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:39.183468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3121d-91a9-48f8-b200-d5fae83e598f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:39.194719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1367/2733] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:29:39.199444 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab20c3-64ee-4655-801e-3aa597ab72b4 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:39.209767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab20c3-64ee-4655-801e-3aa597ab72b4 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:39.214430 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68f5a9-8765-437b-a58f-aa14765191b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:39.215076 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68f5a9-8765-437b-a58f-aa14765191b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:39.215355 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68f5a9-8765-437b-a58f-aa14765191b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:39.220267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68f5a9-8765-437b-a58f-aa14765191b0 admin admin] PATH_INFO: `/v3/projects/0d472a40fdb74beb94fadd2581833a80/users/af554e3e01434d37a54b558574fa6f5d/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:39.230662 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68f5a9-8765-437b-a58f-aa14765191b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d472a40fdb74beb94fadd2581833a80, user_id=af554e3e01434d37a54b558574fa6f5d, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:39.233413 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68f5a9-8765-437b-a58f-aa14765191b0 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:39.349790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1367/2734] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:29:39 2022] PUT /identity/v3/projects/0d472a40fdb74beb94fadd2581833a80/users/af554e3e01434d37a54b558574fa6f5d/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:39.360278 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13b4fb-9318-4bde-891a-762137fd7b83 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:39.367268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13b4fb-9318-4bde-891a-762137fd7b83 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:39.367268 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13b4fb-9318-4bde-891a-762137fd7b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:39.389267 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-ec13b4fb-9318-4bde-891a-762137fd7b83 None None] MFA Rules not processed for user `af554e3e01434d37a54b558574fa6f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:39.390981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ec13b4fb-9318-4bde-891a-762137fd7b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:39.455642 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1368/2735] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:39 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:39.460137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68f5a9-8765-437b-a58f-aa14765191b0 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:39.525863 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68f5a9-8765-437b-a58f-aa14765191b0 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:39.531805 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518f975-c950-4174-9eeb-bbf13ad2b30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:39.532311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518f975-c950-4174-9eeb-bbf13ad2b30e admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:39.532590 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518f975-c950-4174-9eeb-bbf13ad2b30e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:39.532897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518f975-c950-4174-9eeb-bbf13ad2b30e admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:39.533947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518f975-c950-4174-9eeb-bbf13ad2b30e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:39.535439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518f975-c950-4174-9eeb-bbf13ad2b30e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:39.615630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1368/2736] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:29:39 2022] POST /identity/v3/projects => generated 409 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:29:39.619650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13b4fb-9318-4bde-891a-762137fd7b83 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:39.630733 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13b4fb-9318-4bde-891a-762137fd7b83 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:39.656555 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:39.685677 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410122a-77c7-4810-83b9-aeead4ea7c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:39.686316 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410122a-77c7-4810-83b9-aeead4ea7c4c admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:39.686616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410122a-77c7-4810-83b9-aeead4ea7c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:39.686874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410122a-77c7-4810-83b9-aeead4ea7c4c admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:39.688272 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410122a-77c7-4810-83b9-aeead4ea7c4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:39.690210 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410122a-77c7-4810-83b9-aeead4ea7c4c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:39.768853 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1369/2737] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:29:39 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:29:39.773018 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518f975-c950-4174-9eeb-bbf13ad2b30e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:39.783277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518f975-c950-4174-9eeb-bbf13ad2b30e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:39.787767 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed9f4e-f1d6-42a2-8030-6f1124c66c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:39.788350 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed9f4e-f1d6-42a2-8030-6f1124c66c22 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:39.788627 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed9f4e-f1d6-42a2-8030-6f1124c66c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:39.788934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed9f4e-f1d6-42a2-8030-6f1124c66c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:39.789558 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed9f4e-f1d6-42a2-8030-6f1124c66c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:39.791191 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed9f4e-f1d6-42a2-8030-6f1124c66c22 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:39.801616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1369/2738] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:39.805615 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410122a-77c7-4810-83b9-aeead4ea7c4c admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:39.816300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410122a-77c7-4810-83b9-aeead4ea7c4c admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:39.820920 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b22c44-9aa3-4748-ab69-17536ccab4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:39.821554 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b22c44-9aa3-4748-ab69-17536ccab4d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:39.821829 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b22c44-9aa3-4748-ab69-17536ccab4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:39.822107 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b22c44-9aa3-4748-ab69-17536ccab4d3 admin admin] PATH_INFO: `/v3/projects/307e6fa083e54bbd8897dd317d32688f/users/995c9cd8590948d6906a9a4998aa1927/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:39.837065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b22c44-9aa3-4748-ab69-17536ccab4d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=307e6fa083e54bbd8897dd317d32688f, user_id=995c9cd8590948d6906a9a4998aa1927, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:39.839706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b22c44-9aa3-4748-ab69-17536ccab4d3 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:39.899172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1370/2739] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:29:39 2022] PUT /identity/v3/projects/307e6fa083e54bbd8897dd317d32688f/users/995c9cd8590948d6906a9a4998aa1927/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:39.903175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed9f4e-f1d6-42a2-8030-6f1124c66c22 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:39.913230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed9f4e-f1d6-42a2-8030-6f1124c66c22 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:39.971872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026da452-20e5-4e6f-955a-e94412f48a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:39.972454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026da452-20e5-4e6f-955a-e94412f48a91 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:39.972781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026da452-20e5-4e6f-955a-e94412f48a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:39.973058 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026da452-20e5-4e6f-955a-e94412f48a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:39.973700 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026da452-20e5-4e6f-955a-e94412f48a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:39.975282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026da452-20e5-4e6f-955a-e94412f48a91 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:39.985342 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1370/2740] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:39.989514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b22c44-9aa3-4748-ab69-17536ccab4d3 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:39.999576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b22c44-9aa3-4748-ab69-17536ccab4d3 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:40.004690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41307272-72f7-4659-a40b-5aa83a8d1b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:40.005334 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41307272-72f7-4659-a40b-5aa83a8d1b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:40.005619 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41307272-72f7-4659-a40b-5aa83a8d1b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:40.005896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41307272-72f7-4659-a40b-5aa83a8d1b63 admin admin] PATH_INFO: `/v3/projects/307e6fa083e54bbd8897dd317d32688f/users/995c9cd8590948d6906a9a4998aa1927/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:40.007798 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41307272-72f7-4659-a40b-5aa83a8d1b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=307e6fa083e54bbd8897dd317d32688f, user_id=995c9cd8590948d6906a9a4998aa1927, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:40.010439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41307272-72f7-4659-a40b-5aa83a8d1b63 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:40.020464 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1371/2741] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:29:39 2022] PUT /identity/v3/projects/307e6fa083e54bbd8897dd317d32688f/users/995c9cd8590948d6906a9a4998aa1927/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:40.031369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72910532-74c6-4bea-be7d-8b8d58583d81 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:40.031671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72910532-74c6-4bea-be7d-8b8d58583d81 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:40.031951 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72910532-74c6-4bea-be7d-8b8d58583d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:40.060520 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-72910532-74c6-4bea-be7d-8b8d58583d81 None None] MFA Rules not processed for user `995c9cd8590948d6906a9a4998aa1927`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:40.061799 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-72910532-74c6-4bea-be7d-8b8d58583d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:40.078988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:40.131102 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1371/2742] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:40 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:40.142043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41307272-72f7-4659-a40b-5aa83a8d1b63 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:40.143924 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-41307272-72f7-4659-a40b-5aa83a8d1b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:40.200801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41307272-72f7-4659-a40b-5aa83a8d1b63 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:40.204476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a998309-9810-46de-bed3-6f576e4a6d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:40.205105 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a998309-9810-46de-bed3-6f576e4a6d63 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:40.205403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a998309-9810-46de-bed3-6f576e4a6d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:40.205669 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a998309-9810-46de-bed3-6f576e4a6d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:40.214563 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a998309-9810-46de-bed3-6f576e4a6d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:40.219958 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a998309-9810-46de-bed3-6f576e4a6d63 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:40.225285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1372/2743] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:40 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:29:40.594671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72910532-74c6-4bea-be7d-8b8d58583d81 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:40.604324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72910532-74c6-4bea-be7d-8b8d58583d81 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:40.607854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b717a-4ddc-4fa9-bf96-f15a62ba5cea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:40.608451 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b717a-4ddc-4fa9-bf96-f15a62ba5cea service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:40.608758 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b717a-4ddc-4fa9-bf96-f15a62ba5cea service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:40.609837 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b717a-4ddc-4fa9-bf96-f15a62ba5cea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:40.665759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b717a-4ddc-4fa9-bf96-f15a62ba5cea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:40.667465 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b717a-4ddc-4fa9-bf96-f15a62ba5cea service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:40.677080 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1372/2744] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:40 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:29:42.840574 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a998309-9810-46de-bed3-6f576e4a6d63 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:42.842093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5a998309-9810-46de-bed3-6f576e4a6d63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:42.933453 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a998309-9810-46de-bed3-6f576e4a6d63 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:42.939928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cc84f-c854-4802-aefe-8930cad34ac8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:42.940549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cc84f-c854-4802-aefe-8930cad34ac8 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:42.940889 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cc84f-c854-4802-aefe-8930cad34ac8 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:42.941153 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cc84f-c854-4802-aefe-8930cad34ac8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:42.943094 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9f8cc84f-c854-4802-aefe-8930cad34ac8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:42.959736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8cc84f-c854-4802-aefe-8930cad34ac8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:42.961384 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8cc84f-c854-4802-aefe-8930cad34ac8 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:42.974671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1373/2745] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:29:50.673484 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=49 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:51.057976 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b717a-4ddc-4fa9-bf96-f15a62ba5cea service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:51.119206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b717a-4ddc-4fa9-bf96-f15a62ba5cea service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:51.123679 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a95c81-4a80-49ed-811d-7a111142770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:51.124289 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95c81-4a80-49ed-811d-7a111142770b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:51.124568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95c81-4a80-49ed-811d-7a111142770b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:51.124869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95c81-4a80-49ed-811d-7a111142770b admin admin] PATH_INFO: `/v3/users/af554e3e01434d37a54b558574fa6f5d` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:51.126100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a95c81-4a80-49ed-811d-7a111142770b admin admin] RBAC: Authorizing `identity:delete_user(user_id=af554e3e01434d37a54b558574fa6f5d)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:51.128121 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a95c81-4a80-49ed-811d-7a111142770b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:51.366490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b7a95c81-4a80-49ed-811d-7a111142770b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af554e3e01434d37a54b558574fa6f5d'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:51.442367 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b7a95c81-4a80-49ed-811d-7a111142770b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af554e3e01434d37a54b558574fa6f5d'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:51.443137 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b7a95c81-4a80-49ed-811d-7a111142770b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af554e3e01434d37a54b558574fa6f5d'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:51.451276 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b7a95c81-4a80-49ed-811d-7a111142770b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af554e3e01434d37a54b558574fa6f5d'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:51.465024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-b7a95c81-4a80-49ed-811d-7a111142770b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af554e3e01434d37a54b558574fa6f5d'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:51.472286 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1373/2746] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:29:51 2022] DELETE /identity/v3/users/af554e3e01434d37a54b558574fa6f5d => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:51.477051 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843d6f6-0300-4dee-88a8-c687e0ad45c3 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:51.477596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843d6f6-0300-4dee-88a8-c687e0ad45c3 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:51.477596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843d6f6-0300-4dee-88a8-c687e0ad45c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:51.499315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-b843d6f6-0300-4dee-88a8-c687e0ad45c3 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:51.500539 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b843d6f6-0300-4dee-88a8-c687e0ad45c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:51.505200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:51.566413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1374/2747] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:51.576574 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a95c81-4a80-49ed-811d-7a111142770b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:51.578042 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b7a95c81-4a80-49ed-811d-7a111142770b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:51.644786 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a95c81-4a80-49ed-811d-7a111142770b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:51.649616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657107da-296e-4c3b-88b8-8f8b2a0ed893 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:51.649616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657107da-296e-4c3b-88b8-8f8b2a0ed893 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:51.649616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657107da-296e-4c3b-88b8-8f8b2a0ed893 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:51.649616 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657107da-296e-4c3b-88b8-8f8b2a0ed893 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:51.660096 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657107da-296e-4c3b-88b8-8f8b2a0ed893 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:51.661892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657107da-296e-4c3b-88b8-8f8b2a0ed893 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:51.672547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1374/2748] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:29:52.142228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843d6f6-0300-4dee-88a8-c687e0ad45c3 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:52.142228 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b843d6f6-0300-4dee-88a8-c687e0ad45c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:52.154036 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843d6f6-0300-4dee-88a8-c687e0ad45c3 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:52.159621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:52.160446 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:52.160788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:52.161136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] PATH_INFO: `/v3/projects/0d472a40fdb74beb94fadd2581833a80` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:52.162396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d472a40fdb74beb94fadd2581833a80)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:52.164366 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:52.298802 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d472a40fdb74beb94fadd2581833a80', 'request_id': 'req-7c568fc3-bfd8-4659-90b3-51eed9d5e386'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:52.304899 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d472a40fdb74beb94fadd2581833a80', 'request_id': 'req-7c568fc3-bfd8-4659-90b3-51eed9d5e386'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:52.310771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d472a40fdb74beb94fadd2581833a80', 'request_id': 'req-7c568fc3-bfd8-4659-90b3-51eed9d5e386'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:52.446219 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] The token cache is being invalidate because project 0d472a40fdb74beb94fadd2581833a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:29:52.453180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:52.453180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:52.453180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1375/2749] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:29:52 2022] DELETE /identity/v3/projects/0d472a40fdb74beb94fadd2581833a80 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:52.453489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657107da-296e-4c3b-88b8-8f8b2a0ed893 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:52.464565 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657107da-296e-4c3b-88b8-8f8b2a0ed893 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:52.486970 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:52.538330 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f9a82-fd4d-424f-b4eb-a3dc25163c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:52.538974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f9a82-fd4d-424f-b4eb-a3dc25163c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:52.539258 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f9a82-fd4d-424f-b4eb-a3dc25163c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:52.539538 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f9a82-fd4d-424f-b4eb-a3dc25163c58 admin admin] PATH_INFO: `/v3/users/995c9cd8590948d6906a9a4998aa1927` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:52.540721 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f9a82-fd4d-424f-b4eb-a3dc25163c58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=995c9cd8590948d6906a9a4998aa1927)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:52.542892 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f9a82-fd4d-424f-b4eb-a3dc25163c58 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:52.750253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4f4f9a82-fd4d-424f-b4eb-a3dc25163c58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '995c9cd8590948d6906a9a4998aa1927'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:52.842260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4f4f9a82-fd4d-424f-b4eb-a3dc25163c58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '995c9cd8590948d6906a9a4998aa1927'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:52.846859 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4f4f9a82-fd4d-424f-b4eb-a3dc25163c58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '995c9cd8590948d6906a9a4998aa1927'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:52.851142 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4f4f9a82-fd4d-424f-b4eb-a3dc25163c58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '995c9cd8590948d6906a9a4998aa1927'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:52.864014 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4f4f9a82-fd4d-424f-b4eb-a3dc25163c58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '995c9cd8590948d6906a9a4998aa1927'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:52.870186 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1375/2750] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:29:52 2022] DELETE /identity/v3/users/995c9cd8590948d6906a9a4998aa1927 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:53.322019 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:53.323558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:53.337088 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c568fc3-bfd8-4659-90b3-51eed9d5e386 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:53.399308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbbe866-96e3-45fb-8408-9b3a316b7cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:53.399943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbe866-96e3-45fb-8408-9b3a316b7cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:53.400220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbe866-96e3-45fb-8408-9b3a316b7cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:53.400491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbe866-96e3-45fb-8408-9b3a316b7cf5 admin admin] PATH_INFO: `/v3/projects/307e6fa083e54bbd8897dd317d32688f` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:53.402529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbe866-96e3-45fb-8408-9b3a316b7cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=307e6fa083e54bbd8897dd317d32688f)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:53.404461 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbe866-96e3-45fb-8408-9b3a316b7cf5 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:53.504957 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:53.514220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-edbbe866-96e3-45fb-8408-9b3a316b7cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '307e6fa083e54bbd8897dd317d32688f', 'request_id': 'req-edbbe866-96e3-45fb-8408-9b3a316b7cf5'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:53.521042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-edbbe866-96e3-45fb-8408-9b3a316b7cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '307e6fa083e54bbd8897dd317d32688f', 'request_id': 'req-edbbe866-96e3-45fb-8408-9b3a316b7cf5'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:53.526501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-edbbe866-96e3-45fb-8408-9b3a316b7cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '307e6fa083e54bbd8897dd317d32688f', 'request_id': 'req-edbbe866-96e3-45fb-8408-9b3a316b7cf5'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:53.616556 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-edbbe866-96e3-45fb-8408-9b3a316b7cf5 admin admin] The token cache is being invalidate because project 307e6fa083e54bbd8897dd317d32688f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:29:53.616877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-edbbe866-96e3-45fb-8408-9b3a316b7cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:53.617633 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-edbbe866-96e3-45fb-8408-9b3a316b7cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:29:53.619133 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1376/2751] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:29:53 2022] DELETE /identity/v3/projects/307e6fa083e54bbd8897dd317d32688f => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:53.624029 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d1488-6a0e-4a58-8719-24b4f41e44aa None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:53.624490 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d1488-6a0e-4a58-8719-24b4f41e44aa None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:53.624580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d1488-6a0e-4a58-8719-24b4f41e44aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:53.644606 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-111d1488-6a0e-4a58-8719-24b4f41e44aa None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:53.645848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-111d1488-6a0e-4a58-8719-24b4f41e44aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:53.705843 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1376/2752] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:53.713693 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd46285-d73e-4963-b33b-5bbdeeaa9974 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:53.713985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd46285-d73e-4963-b33b-5bbdeeaa9974 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:53.714258 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd46285-d73e-4963-b33b-5bbdeeaa9974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:53.736267 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0bd46285-d73e-4963-b33b-5bbdeeaa9974 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:53.737507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0bd46285-d73e-4963-b33b-5bbdeeaa9974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:53.745451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1377/2753] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:53.749689 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d1488-6a0e-4a58-8719-24b4f41e44aa None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:53.759380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d1488-6a0e-4a58-8719-24b4f41e44aa None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:53.763912 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd674ae8-3906-4775-b858-17c6ab7bfc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:53.764500 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd674ae8-3906-4775-b858-17c6ab7bfc41 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:53.764813 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd674ae8-3906-4775-b858-17c6ab7bfc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:53.765091 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd674ae8-3906-4775-b858-17c6ab7bfc41 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:53.765730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd674ae8-3906-4775-b858-17c6ab7bfc41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:29:53.766041 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd674ae8-3906-4775-b858-17c6ab7bfc41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:53.768156 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd674ae8-3906-4775-b858-17c6ab7bfc41 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:53.779518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1377/2754] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:29:53.784610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd46285-d73e-4963-b33b-5bbdeeaa9974 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:53.793922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd46285-d73e-4963-b33b-5bbdeeaa9974 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:53.798638 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a90d34-2554-439e-b82c-503c1cc2969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:53.799232 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a90d34-2554-439e-b82c-503c1cc2969c admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:53.799513 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a90d34-2554-439e-b82c-503c1cc2969c admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:53.799788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a90d34-2554-439e-b82c-503c1cc2969c admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:53.800410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a90d34-2554-439e-b82c-503c1cc2969c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:53.802336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a90d34-2554-439e-b82c-503c1cc2969c admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:53.892985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1378/2755] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:29:53 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:29:53.897136 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd674ae8-3906-4775-b858-17c6ab7bfc41 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:53.907375 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd674ae8-3906-4775-b858-17c6ab7bfc41 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:53.933950 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:53.970554 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcec3ac-1e91-482f-aed7-c612d484b8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:53.971197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcec3ac-1e91-482f-aed7-c612d484b8af admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:53.971475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcec3ac-1e91-482f-aed7-c612d484b8af admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:53.971747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcec3ac-1e91-482f-aed7-c612d484b8af admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:53.972369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcec3ac-1e91-482f-aed7-c612d484b8af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:53.974327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcec3ac-1e91-482f-aed7-c612d484b8af admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:54.056600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1378/2756] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:29:53 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:29:54.060675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a90d34-2554-439e-b82c-503c1cc2969c admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:54.071435 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a90d34-2554-439e-b82c-503c1cc2969c admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:54.076004 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb5469-d2e2-460f-8b33-cb345510cb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:54.076601 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb5469-d2e2-460f-8b33-cb345510cb76 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:54.076915 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb5469-d2e2-460f-8b33-cb345510cb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:54.077188 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb5469-d2e2-460f-8b33-cb345510cb76 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:54.077830 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb5469-d2e2-460f-8b33-cb345510cb76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:54.080018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb5469-d2e2-460f-8b33-cb345510cb76 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:54.090744 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1379/2757] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:54.095303 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcec3ac-1e91-482f-aed7-c612d484b8af admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:54.107198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcec3ac-1e91-482f-aed7-c612d484b8af admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:54.113293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f563e-1c4f-42ac-a897-edc2e28612a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:54.113902 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f563e-1c4f-42ac-a897-edc2e28612a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:54.114181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f563e-1c4f-42ac-a897-edc2e28612a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:54.114452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f563e-1c4f-42ac-a897-edc2e28612a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:54.115092 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f563e-1c4f-42ac-a897-edc2e28612a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:54.116750 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f563e-1c4f-42ac-a897-edc2e28612a2 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:54.126837 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1379/2758] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:29:54.130904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb5469-d2e2-460f-8b33-cb345510cb76 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:54.140704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb5469-d2e2-460f-8b33-cb345510cb76 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:54.145465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19250448-5c23-49d6-9340-f0e08b1955ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:54.146043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19250448-5c23-49d6-9340-f0e08b1955ed admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:54.146328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19250448-5c23-49d6-9340-f0e08b1955ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:54.146607 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19250448-5c23-49d6-9340-f0e08b1955ed admin admin] PATH_INFO: `/v3/projects/5649ba67cfdc438a91af4ab4367d94bf/users/11d0376c60e94f50836c9a35f1990cbf/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:54.162510 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19250448-5c23-49d6-9340-f0e08b1955ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=5649ba67cfdc438a91af4ab4367d94bf, user_id=11d0376c60e94f50836c9a35f1990cbf, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:54.165218 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19250448-5c23-49d6-9340-f0e08b1955ed admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:54.242731 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1380/2759] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:29:54 2022] PUT /identity/v3/projects/5649ba67cfdc438a91af4ab4367d94bf/users/11d0376c60e94f50836c9a35f1990cbf/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:29:54.254964 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b7bde-1794-4cd6-bdf5-99ef978e2e6e None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:54.255274 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b7bde-1794-4cd6-bdf5-99ef978e2e6e None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:54.255575 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b7bde-1794-4cd6-bdf5-99ef978e2e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:54.285524 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-c58b7bde-1794-4cd6-bdf5-99ef978e2e6e None None] MFA Rules not processed for user `11d0376c60e94f50836c9a35f1990cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:54.286731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-c58b7bde-1794-4cd6-bdf5-99ef978e2e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:54.351209 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1380/2760] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:29:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:29:54.362275 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19250448-5c23-49d6-9340-f0e08b1955ed admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:54.364087 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-19250448-5c23-49d6-9340-f0e08b1955ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:54.420351 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19250448-5c23-49d6-9340-f0e08b1955ed admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:54.423944 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c816dd1-7e55-4dec-8246-f8952efbb880 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:54.424543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c816dd1-7e55-4dec-8246-f8952efbb880 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:54.424852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c816dd1-7e55-4dec-8246-f8952efbb880 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:54.425131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c816dd1-7e55-4dec-8246-f8952efbb880 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:54.433314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c816dd1-7e55-4dec-8246-f8952efbb880 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:54.434727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c816dd1-7e55-4dec-8246-f8952efbb880 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:54.443356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1381/2761] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:29:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:29:55.123252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42812c-2389-4686-b931-b9a79ce2417f None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:55.129088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42812c-2389-4686-b931-b9a79ce2417f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:55.129088 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42812c-2389-4686-b931-b9a79ce2417f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:55.143559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-ea42812c-2389-4686-b931-b9a79ce2417f None None] MFA Rules not processed for user `be379e1a6add42b69e9fdd676efca280`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:29:55.144773 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ea42812c-2389-4686-b931-b9a79ce2417f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:55.204806 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1381/2762] 10.222.0.26 () {58 vars in 967 bytes} [Wed Aug 24 08:29:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:29:55.243745 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c816dd1-7e55-4dec-8246-f8952efbb880 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:29:55.245193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2c816dd1-7e55-4dec-8246-f8952efbb880 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:29:55.314551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:29:55.316963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c816dd1-7e55-4dec-8246-f8952efbb880 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:29:55.322078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb27263-cdc6-48b6-b623-53106c18a5cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b629ad3e6b9f4723b103f9845df2aaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:29:55.322682 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb27263-cdc6-48b6-b623-53106c18a5cd service swift] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:29:55.322963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb27263-cdc6-48b6-b623-53106c18a5cd service swift] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:29:55.323238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb27263-cdc6-48b6-b623-53106c18a5cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:29:55.331411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb27263-cdc6-48b6-b623-53106c18a5cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:29:55.333093 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb27263-cdc6-48b6-b623-53106c18a5cd service swift] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:29:55.341051 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1382/2763] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 24 08:29:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:30:03.393820 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42812c-2389-4686-b931-b9a79ce2417f None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:03.396947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ea42812c-2389-4686-b931-b9a79ce2417f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:03.444170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:30:03.468496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42812c-2389-4686-b931-b9a79ce2417f None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:03.474615 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65f86b-c2b1-4a8b-b1c9-aa730a53a78a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:03.475221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65f86b-c2b1-4a8b-b1c9-aa730a53a78a service placement] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:03.475529 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65f86b-c2b1-4a8b-b1c9-aa730a53a78a service placement] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:03.475828 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65f86b-c2b1-4a8b-b1c9-aa730a53a78a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:03.477568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-1f65f86b-c2b1-4a8b-b1c9-aa730a53a78a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:03.489112 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65f86b-c2b1-4a8b-b1c9-aa730a53a78a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:03.490711 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65f86b-c2b1-4a8b-b1c9-aa730a53a78a service placement] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:03.500457 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1382/2764] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:30:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:30:03.515043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb27263-cdc6-48b6-b623-53106c18a5cd service swift] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:03.517049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-8fb27263-cdc6-48b6-b623-53106c18a5cd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:03.590671 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb27263-cdc6-48b6-b623-53106c18a5cd service swift] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:03.596467 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2af7e-790f-4160-ad81-4a5fe0ea7a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:03.597172 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2af7e-790f-4160-ad81-4a5fe0ea7a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:03.597458 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2af7e-790f-4160-ad81-4a5fe0ea7a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:03.597734 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2af7e-790f-4160-ad81-4a5fe0ea7a04 admin admin] PATH_INFO: `/v3/users/b5600f62677d437ead57459e3cf75e41` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:03.598888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2af7e-790f-4160-ad81-4a5fe0ea7a04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5600f62677d437ead57459e3cf75e41)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:03.600899 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2af7e-790f-4160-ad81-4a5fe0ea7a04 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:03.772555 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5de2af7e-790f-4160-ad81-4a5fe0ea7a04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5600f62677d437ead57459e3cf75e41'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:03.779687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5de2af7e-790f-4160-ad81-4a5fe0ea7a04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5600f62677d437ead57459e3cf75e41'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:03.851472 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5de2af7e-790f-4160-ad81-4a5fe0ea7a04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5600f62677d437ead57459e3cf75e41'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:03.852137 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5de2af7e-790f-4160-ad81-4a5fe0ea7a04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5600f62677d437ead57459e3cf75e41'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:03.865169 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5de2af7e-790f-4160-ad81-4a5fe0ea7a04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5600f62677d437ead57459e3cf75e41'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:03.870949 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1383/2765] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:30:03 2022] DELETE /identity/v3/users/b5600f62677d437ead57459e3cf75e41 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:30:04.112155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65f86b-c2b1-4a8b-b1c9-aa730a53a78a service placement] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:04.123105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65f86b-c2b1-4a8b-b1c9-aa730a53a78a service placement] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:04.181981 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:04.182609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:04.182893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:04.183171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] PATH_INFO: `/v3/projects/19a004d608f2454cb5e9e65eef5ed740` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:04.184353 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a004d608f2454cb5e9e65eef5ed740)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:04.186308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:04.288609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a004d608f2454cb5e9e65eef5ed740', 'request_id': 'req-a70193e9-490b-4762-bb61-ab4b74213c82'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.295147 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a004d608f2454cb5e9e65eef5ed740', 'request_id': 'req-a70193e9-490b-4762-bb61-ab4b74213c82'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.349927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a004d608f2454cb5e9e65eef5ed740', 'request_id': 'req-a70193e9-490b-4762-bb61-ab4b74213c82'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.360858 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] The token cache is being invalidate because project 19a004d608f2454cb5e9e65eef5ed740 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:30:04.361077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.361704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.363254 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1383/2766] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:30:04 2022] DELETE /identity/v3/projects/19a004d608f2454cb5e9e65eef5ed740 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:30:04.367264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2af7e-790f-4160-ad81-4a5fe0ea7a04 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:04.368685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5de2af7e-790f-4160-ad81-4a5fe0ea7a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:04.381950 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2af7e-790f-4160-ad81-4a5fe0ea7a04 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:04.447468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039a1c6-7394-4d77-9c32-366cdcb041e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:04.448124 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039a1c6-7394-4d77-9c32-366cdcb041e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:04.448410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039a1c6-7394-4d77-9c32-366cdcb041e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:04.448577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:04.449035 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039a1c6-7394-4d77-9c32-366cdcb041e6 admin admin] PATH_INFO: `/v3/users/2a55fd90c7c54f5bb842707b84a4ed9c` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:04.449918 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039a1c6-7394-4d77-9c32-366cdcb041e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a55fd90c7c54f5bb842707b84a4ed9c)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:04.450030 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:04.452737 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039a1c6-7394-4d77-9c32-366cdcb041e6 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:04.465595 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:30:04.466274 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70193e9-490b-4762-bb61-ab4b74213c82 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:04.472818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb75ae5-bf1a-4f8b-83fb-df582321735f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:04.473460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb75ae5-bf1a-4f8b-83fb-df582321735f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:04.473739 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb75ae5-bf1a-4f8b-83fb-df582321735f admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:04.474139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb75ae5-bf1a-4f8b-83fb-df582321735f admin admin] PATH_INFO: `/v3/users/11d0376c60e94f50836c9a35f1990cbf` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:04.475321 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb75ae5-bf1a-4f8b-83fb-df582321735f admin admin] RBAC: Authorizing `identity:delete_user(user_id=11d0376c60e94f50836c9a35f1990cbf)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:04.477425 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb75ae5-bf1a-4f8b-83fb-df582321735f admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:04.647398 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1039a1c6-7394-4d77-9c32-366cdcb041e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a55fd90c7c54f5bb842707b84a4ed9c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.653903 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1039a1c6-7394-4d77-9c32-366cdcb041e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a55fd90c7c54f5bb842707b84a4ed9c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.675987 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ffb75ae5-bf1a-4f8b-83fb-df582321735f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11d0376c60e94f50836c9a35f1990cbf'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.742307 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1039a1c6-7394-4d77-9c32-366cdcb041e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a55fd90c7c54f5bb842707b84a4ed9c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.742972 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1039a1c6-7394-4d77-9c32-366cdcb041e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a55fd90c7c54f5bb842707b84a4ed9c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.756127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1039a1c6-7394-4d77-9c32-366cdcb041e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a55fd90c7c54f5bb842707b84a4ed9c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.762091 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1384/2767] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:30:04 2022] DELETE /identity/v3/users/2a55fd90c7c54f5bb842707b84a4ed9c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:30:04.805659 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ffb75ae5-bf1a-4f8b-83fb-df582321735f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11d0376c60e94f50836c9a35f1990cbf'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.806309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ffb75ae5-bf1a-4f8b-83fb-df582321735f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11d0376c60e94f50836c9a35f1990cbf'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.808884 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:30:04.815453 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ffb75ae5-bf1a-4f8b-83fb-df582321735f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11d0376c60e94f50836c9a35f1990cbf'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.827308 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-ffb75ae5-bf1a-4f8b-83fb-df582321735f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11d0376c60e94f50836c9a35f1990cbf'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:04.833645 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1384/2768] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:30:04 2022] DELETE /identity/v3/users/11d0376c60e94f50836c9a35f1990cbf => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:30:04.841006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef413d-b690-461d-a8ee-ed6a270d4914 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:04.841297 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef413d-b690-461d-a8ee-ed6a270d4914 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:04.841575 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef413d-b690-461d-a8ee-ed6a270d4914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:04.861970 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-d2ef413d-b690-461d-a8ee-ed6a270d4914 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:30:04.865295 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d2ef413d-b690-461d-a8ee-ed6a270d4914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:04.935439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1385/2769] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:30:04.945864 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb75ae5-bf1a-4f8b-83fb-df582321735f admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:04.947278 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-ffb75ae5-bf1a-4f8b-83fb-df582321735f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:05.005147 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb75ae5-bf1a-4f8b-83fb-df582321735f admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:05.008870 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb2ca0-a5c8-4787-ac84-b077be59fbfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:05.009474 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb2ca0-a5c8-4787-ac84-b077be59fbfd service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:05.009759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb2ca0-a5c8-4787-ac84-b077be59fbfd service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:05.010035 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb2ca0-a5c8-4787-ac84-b077be59fbfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:05.019000 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb2ca0-a5c8-4787-ac84-b077be59fbfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:05.020435 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb2ca0-a5c8-4787-ac84-b077be59fbfd service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:05.029815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1385/2770] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:30:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:30:05.035819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef413d-b690-461d-a8ee-ed6a270d4914 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:05.037501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d2ef413d-b690-461d-a8ee-ed6a270d4914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:05.049833 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef413d-b690-461d-a8ee-ed6a270d4914 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:05.054593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37916db3-d16b-4293-ba04-4b35efd6ae88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:05.055260 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37916db3-d16b-4293-ba04-4b35efd6ae88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:05.055544 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37916db3-d16b-4293-ba04-4b35efd6ae88 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:05.055811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37916db3-d16b-4293-ba04-4b35efd6ae88 admin admin] PATH_INFO: `/v3/projects/3c8624cf5bb7465f8214347ac02be54e` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:05.056975 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37916db3-d16b-4293-ba04-4b35efd6ae88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c8624cf5bb7465f8214347ac02be54e)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:05.058909 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37916db3-d16b-4293-ba04-4b35efd6ae88 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:05.162045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-37916db3-d16b-4293-ba04-4b35efd6ae88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c8624cf5bb7465f8214347ac02be54e', 'request_id': 'req-37916db3-d16b-4293-ba04-4b35efd6ae88'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:05.169722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-37916db3-d16b-4293-ba04-4b35efd6ae88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c8624cf5bb7465f8214347ac02be54e', 'request_id': 'req-37916db3-d16b-4293-ba04-4b35efd6ae88'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:05.176415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-37916db3-d16b-4293-ba04-4b35efd6ae88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c8624cf5bb7465f8214347ac02be54e', 'request_id': 'req-37916db3-d16b-4293-ba04-4b35efd6ae88'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:05.272959 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-37916db3-d16b-4293-ba04-4b35efd6ae88 admin admin] The token cache is being invalidate because project 3c8624cf5bb7465f8214347ac02be54e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:30:05.273129 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-37916db3-d16b-4293-ba04-4b35efd6ae88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:05.273846 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-37916db3-d16b-4293-ba04-4b35efd6ae88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:05.275594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1386/2771] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:30:05 2022] DELETE /identity/v3/projects/3c8624cf5bb7465f8214347ac02be54e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:30:05.495279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb2ca0-a5c8-4787-ac84-b077be59fbfd service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:05.497470 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-80bb2ca0-a5c8-4787-ac84-b077be59fbfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:05.511687 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb2ca0-a5c8-4787-ac84-b077be59fbfd service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:05.576624 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39177349-bc73-47fd-9e8b-20d00cfa9d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:05.577277 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39177349-bc73-47fd-9e8b-20d00cfa9d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:05.577557 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39177349-bc73-47fd-9e8b-20d00cfa9d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:05.577833 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39177349-bc73-47fd-9e8b-20d00cfa9d17 admin admin] PATH_INFO: `/v3/projects/5649ba67cfdc438a91af4ab4367d94bf` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:05.579776 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39177349-bc73-47fd-9e8b-20d00cfa9d17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5649ba67cfdc438a91af4ab4367d94bf)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:05.581724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39177349-bc73-47fd-9e8b-20d00cfa9d17 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:05.701671 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-39177349-bc73-47fd-9e8b-20d00cfa9d17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5649ba67cfdc438a91af4ab4367d94bf', 'request_id': 'req-39177349-bc73-47fd-9e8b-20d00cfa9d17'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:05.708433 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-39177349-bc73-47fd-9e8b-20d00cfa9d17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5649ba67cfdc438a91af4ab4367d94bf', 'request_id': 'req-39177349-bc73-47fd-9e8b-20d00cfa9d17'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:05.782871 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-39177349-bc73-47fd-9e8b-20d00cfa9d17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5649ba67cfdc438a91af4ab4367d94bf', 'request_id': 'req-39177349-bc73-47fd-9e8b-20d00cfa9d17'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:05.793893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-39177349-bc73-47fd-9e8b-20d00cfa9d17 admin admin] The token cache is being invalidate because project 5649ba67cfdc438a91af4ab4367d94bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:30:05.794188 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-39177349-bc73-47fd-9e8b-20d00cfa9d17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:05.794822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-39177349-bc73-47fd-9e8b-20d00cfa9d17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:05.798551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1386/2772] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:30:05 2022] DELETE /identity/v3/projects/5649ba67cfdc438a91af4ab4367d94bf => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:30:05.801712 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5b510-dbfd-4fb7-8fc0-c87980c71bc5 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:05.802157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5b510-dbfd-4fb7-8fc0-c87980c71bc5 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:05.802266 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5b510-dbfd-4fb7-8fc0-c87980c71bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:05.822844 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-f2c5b510-dbfd-4fb7-8fc0-c87980c71bc5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:30:05.824043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f2c5b510-dbfd-4fb7-8fc0-c87980c71bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:05.832842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:30:05.881832 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1387/2773] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:30:05.889446 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0e56b-813d-4c0d-8e7c-e73f4a996d41 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:05.889844 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0e56b-813d-4c0d-8e7c-e73f4a996d41 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:05.890198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0e56b-813d-4c0d-8e7c-e73f4a996d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:05.906564 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:30:05.912054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-61f0e56b-813d-4c0d-8e7c-e73f4a996d41 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:30:05.913353 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-61f0e56b-813d-4c0d-8e7c-e73f4a996d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:05.921617 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1387/2774] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:30:05.926128 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5b510-dbfd-4fb7-8fc0-c87980c71bc5 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:05.935859 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5b510-dbfd-4fb7-8fc0-c87980c71bc5 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:05.941970 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427be037-849f-4b7d-a69f-2d6ea2b80d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:05.942573 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427be037-849f-4b7d-a69f-2d6ea2b80d71 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:05.942852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427be037-849f-4b7d-a69f-2d6ea2b80d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:05.943153 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427be037-849f-4b7d-a69f-2d6ea2b80d71 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:05.943796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427be037-849f-4b7d-a69f-2d6ea2b80d71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:30:05.944100 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427be037-849f-4b7d-a69f-2d6ea2b80d71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:05.945709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427be037-849f-4b7d-a69f-2d6ea2b80d71 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:05.962971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1388/2775] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:30:05.964074 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0e56b-813d-4c0d-8e7c-e73f4a996d41 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:05.974900 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0e56b-813d-4c0d-8e7c-e73f4a996d41 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:05.979665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e85e20-a4b3-487b-a364-25b71eb72405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:05.980192 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e85e20-a4b3-487b-a364-25b71eb72405 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:05.980473 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e85e20-a4b3-487b-a364-25b71eb72405 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:05.980784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e85e20-a4b3-487b-a364-25b71eb72405 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:05.981420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e85e20-a4b3-487b-a364-25b71eb72405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:05.983298 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e85e20-a4b3-487b-a364-25b71eb72405 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:06.068869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1388/2776] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:30:05 2022] POST /identity/v3/projects => generated 361 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:30:06.072995 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427be037-849f-4b7d-a69f-2d6ea2b80d71 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:06.082871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427be037-849f-4b7d-a69f-2d6ea2b80d71 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:06.137972 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f3413-d56b-4e9a-b66f-a32d6b8a1dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:06.138565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f3413-d56b-4e9a-b66f-a32d6b8a1dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:06.138843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f3413-d56b-4e9a-b66f-a32d6b8a1dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:06.139113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f3413-d56b-4e9a-b66f-a32d6b8a1dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:06.143187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f3413-d56b-4e9a-b66f-a32d6b8a1dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:06.143187 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f3413-d56b-4e9a-b66f-a32d6b8a1dd2 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:06.217005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1389/2777] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:30:06 2022] POST /identity/v3/users => generated 281 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:30:06.221900 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e85e20-a4b3-487b-a364-25b71eb72405 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:06.231930 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e85e20-a4b3-487b-a364-25b71eb72405 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:06.236690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39ef82-f5ef-41d4-a658-4d434eaf0647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:06.237285 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39ef82-f5ef-41d4-a658-4d434eaf0647 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:06.237566 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39ef82-f5ef-41d4-a658-4d434eaf0647 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:06.237840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39ef82-f5ef-41d4-a658-4d434eaf0647 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:06.238549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39ef82-f5ef-41d4-a658-4d434eaf0647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:06.240140 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39ef82-f5ef-41d4-a658-4d434eaf0647 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:06.250444 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1389/2778] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:30:06.261801 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f3413-d56b-4e9a-b66f-a32d6b8a1dd2 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:06.272332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f3413-d56b-4e9a-b66f-a32d6b8a1dd2 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:06.277496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3361c8-a364-4a19-ac9d-b8430ada93ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:06.278514 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3361c8-a364-4a19-ac9d-b8430ada93ca admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:06.278804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3361c8-a364-4a19-ac9d-b8430ada93ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:06.279085 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3361c8-a364-4a19-ac9d-b8430ada93ca admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:06.279736 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3361c8-a364-4a19-ac9d-b8430ada93ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:06.282038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3361c8-a364-4a19-ac9d-b8430ada93ca admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:06.293439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1390/2779] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:30:06.297836 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39ef82-f5ef-41d4-a658-4d434eaf0647 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:06.308166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39ef82-f5ef-41d4-a658-4d434eaf0647 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:06.313120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da70228-695f-4b52-95cf-91fec44eb277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:06.313765 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da70228-695f-4b52-95cf-91fec44eb277 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:06.314050 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da70228-695f-4b52-95cf-91fec44eb277 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:06.314327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da70228-695f-4b52-95cf-91fec44eb277 admin admin] PATH_INFO: `/v3/projects/ac8fae0777f64dd3b9947af1541fa734/users/d23c23950d744778a48bf42e5964240c/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:06.329464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da70228-695f-4b52-95cf-91fec44eb277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac8fae0777f64dd3b9947af1541fa734, user_id=d23c23950d744778a48bf42e5964240c, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:06.332246 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da70228-695f-4b52-95cf-91fec44eb277 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:06.428456 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1390/2780] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:30:06 2022] PUT /identity/v3/projects/ac8fae0777f64dd3b9947af1541fa734/users/d23c23950d744778a48bf42e5964240c/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:30:06.438793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e824c5-b471-4e54-9582-8b7b21d2c12d None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:06.439082 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e824c5-b471-4e54-9582-8b7b21d2c12d None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:06.439383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e824c5-b471-4e54-9582-8b7b21d2c12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:06.467048 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-54e824c5-b471-4e54-9582-8b7b21d2c12d None None] MFA Rules not processed for user `d23c23950d744778a48bf42e5964240c`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:30:06.468261 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-54e824c5-b471-4e54-9582-8b7b21d2c12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:06.532098 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1391/2781] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:30:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:30:06.564989 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da70228-695f-4b52-95cf-91fec44eb277 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:06.566825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9da70228-695f-4b52-95cf-91fec44eb277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:06.623810 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da70228-695f-4b52-95cf-91fec44eb277 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:06.628125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf2dce-9f71-42c0-80a7-1496cd69d2e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:06.628748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf2dce-9f71-42c0-80a7-1496cd69d2e0 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:06.629027 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf2dce-9f71-42c0-80a7-1496cd69d2e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:06.629297 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf2dce-9f71-42c0-80a7-1496cd69d2e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:06.637460 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf2dce-9f71-42c0-80a7-1496cd69d2e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:06.638897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf2dce-9f71-42c0-80a7-1496cd69d2e0 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:06.648315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1391/2782] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:30:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:30:06.664002 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e824c5-b471-4e54-9582-8b7b21d2c12d None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:06.674179 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:30:06.725872 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e824c5-b471-4e54-9582-8b7b21d2c12d None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:06.730404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf38f78-1f56-432a-9a9b-b19b819fc47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:06.731018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf38f78-1f56-432a-9a9b-b19b819fc47d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:06.731298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf38f78-1f56-432a-9a9b-b19b819fc47d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:06.731647 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf38f78-1f56-432a-9a9b-b19b819fc47d admin admin] PATH_INFO: `/v3/users/d23c23950d744778a48bf42e5964240c` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:06.732928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf38f78-1f56-432a-9a9b-b19b819fc47d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d23c23950d744778a48bf42e5964240c)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:06.735481 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf38f78-1f56-432a-9a9b-b19b819fc47d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:06.889792 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bbf38f78-1f56-432a-9a9b-b19b819fc47d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd23c23950d744778a48bf42e5964240c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:06.897052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bbf38f78-1f56-432a-9a9b-b19b819fc47d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd23c23950d744778a48bf42e5964240c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:06.953196 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bbf38f78-1f56-432a-9a9b-b19b819fc47d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd23c23950d744778a48bf42e5964240c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:06.954592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bbf38f78-1f56-432a-9a9b-b19b819fc47d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd23c23950d744778a48bf42e5964240c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:06.966790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bbf38f78-1f56-432a-9a9b-b19b819fc47d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd23c23950d744778a48bf42e5964240c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:06.972859 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1392/2783] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:30:06 2022] DELETE /identity/v3/users/d23c23950d744778a48bf42e5964240c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:30:06.977349 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6280b-c412-4596-9873-0d7ad9df7c78 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:06.977636 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6280b-c412-4596-9873-0d7ad9df7c78 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:06.977910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6280b-c412-4596-9873-0d7ad9df7c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:06.997331 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-47d6280b-c412-4596-9873-0d7ad9df7c78 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:30:06.998602 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-47d6280b-c412-4596-9873-0d7ad9df7c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:07.061713 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1392/2784] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:30:07.072411 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf38f78-1f56-432a-9a9b-b19b819fc47d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:07.073812 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-bbf38f78-1f56-432a-9a9b-b19b819fc47d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:07.132380 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf38f78-1f56-432a-9a9b-b19b819fc47d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:07.135965 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505ec64-fe5b-4723-9df3-f190d0b87b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:07.136558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505ec64-fe5b-4723-9df3-f190d0b87b13 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:07.136862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505ec64-fe5b-4723-9df3-f190d0b87b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:07.137131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505ec64-fe5b-4723-9df3-f190d0b87b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:07.146110 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505ec64-fe5b-4723-9df3-f190d0b87b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:07.147467 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505ec64-fe5b-4723-9df3-f190d0b87b13 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:07.156783 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1393/2785] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:30:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:30:07.572293 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6280b-c412-4596-9873-0d7ad9df7c78 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:07.573790 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-47d6280b-c412-4596-9873-0d7ad9df7c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:07.586133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6280b-c412-4596-9873-0d7ad9df7c78 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:07.590899 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:07.591531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:07.591817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:07.592230 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] PATH_INFO: `/v3/projects/ac8fae0777f64dd3b9947af1541fa734` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:07.594269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac8fae0777f64dd3b9947af1541fa734)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:07.596236 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:07.690967 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:30:07.696999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac8fae0777f64dd3b9947af1541fa734', 'request_id': 'req-e04056b2-cca0-426d-9876-60c5463b3473'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:07.703515 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac8fae0777f64dd3b9947af1541fa734', 'request_id': 'req-e04056b2-cca0-426d-9876-60c5463b3473'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:07.807129 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac8fae0777f64dd3b9947af1541fa734', 'request_id': 'req-e04056b2-cca0-426d-9876-60c5463b3473'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:07.818160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] The token cache is being invalidate because project ac8fae0777f64dd3b9947af1541fa734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:30:07.818693 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:07.819663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:30:07.821271 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1393/2786] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:30:07 2022] DELETE /identity/v3/projects/ac8fae0777f64dd3b9947af1541fa734 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:30:17.152459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=75,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:30:17.876077 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=9,keystone:DELETE=4,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:30:30.368719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505ec64-fe5b-4723-9df3-f190d0b87b13 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:30.370132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9505ec64-fe5b-4723-9df3-f190d0b87b13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:30.490686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505ec64-fe5b-4723-9df3-f190d0b87b13 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:30.552902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0acab-af84-4cc1-9166-4c33dc83c4d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:30.553512 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0acab-af84-4cc1-9166-4c33dc83c4d2 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:30.553789 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0acab-af84-4cc1-9166-4c33dc83c4d2 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:30.554060 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0acab-af84-4cc1-9166-4c33dc83c4d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:30.555794 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-dbe0acab-af84-4cc1-9166-4c33dc83c4d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:30.603270 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:30:30.634692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0acab-af84-4cc1-9166-4c33dc83c4d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:30.636193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0acab-af84-4cc1-9166-4c33dc83c4d2 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:30.654555 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1394/2787] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:30:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:30:40.652054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=19 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:30:41.234327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:30:41.235784 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:41.351913 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04056b2-cca0-426d-9876-60c5463b3473 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:30:41.357144 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181b882-2eec-4a3e-9c5c-87013c131226 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:30:41.357750 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181b882-2eec-4a3e-9c5c-87013c131226 service placement] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:30:41.358030 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181b882-2eec-4a3e-9c5c-87013c131226 service placement] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:30:41.358475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181b882-2eec-4a3e-9c5c-87013c131226 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:30:41.360252 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f181b882-2eec-4a3e-9c5c-87013c131226 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:30:41.371433 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181b882-2eec-4a3e-9c5c-87013c131226 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:30:41.373044 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181b882-2eec-4a3e-9c5c-87013c131226 service placement] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:30:41.383185 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1394/2788] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:30:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:30:51.379049 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=52 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:30.597206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0acab-af84-4cc1-9166-4c33dc83c4d2 service nova] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:30.598678 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-dbe0acab-af84-4cc1-9166-4c33dc83c4d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:30.665617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0acab-af84-4cc1-9166-4c33dc83c4d2 service nova] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:30.670211 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2194c42-ba26-4a63-b113-a1eb02302a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:30.670845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2194c42-ba26-4a63-b113-a1eb02302a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:30.671127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2194c42-ba26-4a63-b113-a1eb02302a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:30.671405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2194c42-ba26-4a63-b113-a1eb02302a05 admin admin] PATH_INFO: `/v3/users/5fb0b775f9ee4a44a736346a173bdd49` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:30.672519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2194c42-ba26-4a63-b113-a1eb02302a05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb0b775f9ee4a44a736346a173bdd49)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:30.674499 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2194c42-ba26-4a63-b113-a1eb02302a05 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:30.890905 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b2194c42-ba26-4a63-b113-a1eb02302a05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb0b775f9ee4a44a736346a173bdd49'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:30.897631 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b2194c42-ba26-4a63-b113-a1eb02302a05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb0b775f9ee4a44a736346a173bdd49'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:30.963220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b2194c42-ba26-4a63-b113-a1eb02302a05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb0b775f9ee4a44a736346a173bdd49'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:30.963840 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b2194c42-ba26-4a63-b113-a1eb02302a05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb0b775f9ee4a44a736346a173bdd49'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:30.976477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b2194c42-ba26-4a63-b113-a1eb02302a05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb0b775f9ee4a44a736346a173bdd49'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:30.982552 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1395/2789] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:32:30 2022] DELETE /identity/v3/users/5fb0b775f9ee4a44a736346a173bdd49 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:30.987861 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16111839-2c78-4c5d-939b-687a450aec52 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:30.988346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16111839-2c78-4c5d-939b-687a450aec52 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:30.988445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16111839-2c78-4c5d-939b-687a450aec52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:31.010402 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-16111839-2c78-4c5d-939b-687a450aec52 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:31.011630 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-16111839-2c78-4c5d-939b-687a450aec52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:31.077336 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1395/2790] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:31.088441 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2194c42-ba26-4a63-b113-a1eb02302a05 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:31.089871 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b2194c42-ba26-4a63-b113-a1eb02302a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:31.155525 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2194c42-ba26-4a63-b113-a1eb02302a05 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:31.159163 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a060f-9d8c-41fa-8c64-f5368eea3094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:31.159764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a060f-9d8c-41fa-8c64-f5368eea3094 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:31.160046 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a060f-9d8c-41fa-8c64-f5368eea3094 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:31.160319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a060f-9d8c-41fa-8c64-f5368eea3094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:31.169598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a060f-9d8c-41fa-8c64-f5368eea3094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:31.171706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a060f-9d8c-41fa-8c64-f5368eea3094 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:31.180991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1396/2791] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:32:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:32:31.842949 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16111839-2c78-4c5d-939b-687a450aec52 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:31.844462 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-16111839-2c78-4c5d-939b-687a450aec52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:31.856839 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16111839-2c78-4c5d-939b-687a450aec52 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:31.862704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9c08b-0391-4175-94c4-ce77bd7a308e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:31.863341 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9c08b-0391-4175-94c4-ce77bd7a308e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:31.863622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9c08b-0391-4175-94c4-ce77bd7a308e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:31.863900 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9c08b-0391-4175-94c4-ce77bd7a308e admin admin] PATH_INFO: `/v3/projects/0ed82d8ed9a4431f8528213ac5e91c7e` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:31.866172 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9c08b-0391-4175-94c4-ce77bd7a308e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ed82d8ed9a4431f8528213ac5e91c7e)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:31.868810 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9c08b-0391-4175-94c4-ce77bd7a308e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:31.981960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-82b9c08b-0391-4175-94c4-ce77bd7a308e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ed82d8ed9a4431f8528213ac5e91c7e', 'request_id': 'req-82b9c08b-0391-4175-94c4-ce77bd7a308e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:31.989989 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-82b9c08b-0391-4175-94c4-ce77bd7a308e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ed82d8ed9a4431f8528213ac5e91c7e', 'request_id': 'req-82b9c08b-0391-4175-94c4-ce77bd7a308e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:32.051609 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-82b9c08b-0391-4175-94c4-ce77bd7a308e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ed82d8ed9a4431f8528213ac5e91c7e', 'request_id': 'req-82b9c08b-0391-4175-94c4-ce77bd7a308e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:32.065973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-82b9c08b-0391-4175-94c4-ce77bd7a308e admin admin] The token cache is being invalidate because project 0ed82d8ed9a4431f8528213ac5e91c7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:32:32.066495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-82b9c08b-0391-4175-94c4-ce77bd7a308e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:32.067495 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-82b9c08b-0391-4175-94c4-ce77bd7a308e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:32.070126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1396/2792] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:32:31 2022] DELETE /identity/v3/projects/0ed82d8ed9a4431f8528213ac5e91c7e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:32.076045 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3d7ad-5b01-4a27-a61c-74a7e1d137f0 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:32.077162 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3d7ad-5b01-4a27-a61c-74a7e1d137f0 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:32.077477 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3d7ad-5b01-4a27-a61c-74a7e1d137f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:32.097178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-92d3d7ad-5b01-4a27-a61c-74a7e1d137f0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:32.098405 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-92d3d7ad-5b01-4a27-a61c-74a7e1d137f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:32.114960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:32.157082 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1397/2793] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:32.165234 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f7b0a-124c-475c-887f-3b662f71118f None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:32.165523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f7b0a-124c-475c-887f-3b662f71118f None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:32.165822 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f7b0a-124c-475c-887f-3b662f71118f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:32.186798 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-7e3f7b0a-124c-475c-887f-3b662f71118f None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:32.188029 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-7e3f7b0a-124c-475c-887f-3b662f71118f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:32.196423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1397/2794] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:32.200704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3d7ad-5b01-4a27-a61c-74a7e1d137f0 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:32.210936 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3d7ad-5b01-4a27-a61c-74a7e1d137f0 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:32.215431 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684f275-f708-4f07-bb26-a032019ca659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:32.216052 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684f275-f708-4f07-bb26-a032019ca659 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:32.216521 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684f275-f708-4f07-bb26-a032019ca659 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:32.217077 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684f275-f708-4f07-bb26-a032019ca659 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:32.217837 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684f275-f708-4f07-bb26-a032019ca659 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:32:32.218480 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684f275-f708-4f07-bb26-a032019ca659 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:32.220071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684f275-f708-4f07-bb26-a032019ca659 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:32.231601 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1398/2795] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:32:32.236116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f7b0a-124c-475c-887f-3b662f71118f None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:32.245321 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f7b0a-124c-475c-887f-3b662f71118f None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:32.250812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926b09f-5f73-4cde-9139-8c8ef135a857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:32.251459 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926b09f-5f73-4cde-9139-8c8ef135a857 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:32.251738 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926b09f-5f73-4cde-9139-8c8ef135a857 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:32.252009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926b09f-5f73-4cde-9139-8c8ef135a857 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:32.252629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926b09f-5f73-4cde-9139-8c8ef135a857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:32.254600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926b09f-5f73-4cde-9139-8c8ef135a857 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:32.315797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1398/2796] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:32:32 2022] POST /identity/v3/projects => generated 401 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:32:32.320013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684f275-f708-4f07-bb26-a032019ca659 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:32.331206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684f275-f708-4f07-bb26-a032019ca659 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:32.387743 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834cb73-c16e-4b06-ab29-d8bc5de06fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:32.388342 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834cb73-c16e-4b06-ab29-d8bc5de06fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:32.388622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834cb73-c16e-4b06-ab29-d8bc5de06fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:32.388955 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834cb73-c16e-4b06-ab29-d8bc5de06fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:32.389706 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834cb73-c16e-4b06-ab29-d8bc5de06fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:32.391621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834cb73-c16e-4b06-ab29-d8bc5de06fc1 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:32.480237 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1399/2797] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:32:32 2022] POST /identity/v3/users => generated 301 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:32:32.485159 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926b09f-5f73-4cde-9139-8c8ef135a857 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:32.495634 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926b09f-5f73-4cde-9139-8c8ef135a857 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:32.501117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf8fdc-ff50-4351-a6c4-0ae1577770f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:32.501719 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf8fdc-ff50-4351-a6c4-0ae1577770f1 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:32.502022 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf8fdc-ff50-4351-a6c4-0ae1577770f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:32.502284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf8fdc-ff50-4351-a6c4-0ae1577770f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:32.502947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf8fdc-ff50-4351-a6c4-0ae1577770f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:32.504549 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf8fdc-ff50-4351-a6c4-0ae1577770f1 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:32.515197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1399/2798] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:32.519237 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834cb73-c16e-4b06-ab29-d8bc5de06fc1 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:32.529355 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834cb73-c16e-4b06-ab29-d8bc5de06fc1 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:32.536742 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f486c0f-1d33-4dec-a605-5583121388db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:32.537343 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f486c0f-1d33-4dec-a605-5583121388db admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:32.537621 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f486c0f-1d33-4dec-a605-5583121388db admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:32.537910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f486c0f-1d33-4dec-a605-5583121388db admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:32.538527 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f486c0f-1d33-4dec-a605-5583121388db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:32.540114 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f486c0f-1d33-4dec-a605-5583121388db admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:32.551882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1400/2799] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:32.556005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf8fdc-ff50-4351-a6c4-0ae1577770f1 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:32.567025 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf8fdc-ff50-4351-a6c4-0ae1577770f1 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:32.571707 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36990b-ca43-4073-9d9a-407928bbee5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:32.572353 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36990b-ca43-4073-9d9a-407928bbee5d admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:32.572633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36990b-ca43-4073-9d9a-407928bbee5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:32.572942 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36990b-ca43-4073-9d9a-407928bbee5d admin admin] PATH_INFO: `/v3/projects/46d2c48079ed4ab085cccbcedc7bffcb/users/47f4b130b63a41629017fa9632472b9a/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:32.587107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36990b-ca43-4073-9d9a-407928bbee5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=46d2c48079ed4ab085cccbcedc7bffcb, user_id=47f4b130b63a41629017fa9632472b9a, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:32.589786 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36990b-ca43-4073-9d9a-407928bbee5d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:32.660707 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1400/2800] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:32:32 2022] PUT /identity/v3/projects/46d2c48079ed4ab085cccbcedc7bffcb/users/47f4b130b63a41629017fa9632472b9a/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:32.672340 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c036d37-fe94-4205-b027-a6a3ccdb29a3 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:32.672624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c036d37-fe94-4205-b027-a6a3ccdb29a3 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:32.672926 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c036d37-fe94-4205-b027-a6a3ccdb29a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:32.701291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-0c036d37-fe94-4205-b027-a6a3ccdb29a3 None None] MFA Rules not processed for user `47f4b130b63a41629017fa9632472b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:32.702596 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0c036d37-fe94-4205-b027-a6a3ccdb29a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:32.765887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1401/2801] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:32 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:32.770404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36990b-ca43-4073-9d9a-407928bbee5d admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:32.791793 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:32.832577 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36990b-ca43-4073-9d9a-407928bbee5d admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:32.837428 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a4997-3bed-40f5-a687-88c065682e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:32.838031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a4997-3bed-40f5-a687-88c065682e16 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:32.838323 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a4997-3bed-40f5-a687-88c065682e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:32.838610 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a4997-3bed-40f5-a687-88c065682e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:32.839245 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a4997-3bed-40f5-a687-88c065682e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:32.841176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a4997-3bed-40f5-a687-88c065682e16 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:32.955991 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1401/2802] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:32:32 2022] POST /identity/v3/projects => generated 405 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:32:32.960096 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c036d37-fe94-4205-b027-a6a3ccdb29a3 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:32.971216 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c036d37-fe94-4205-b027-a6a3ccdb29a3 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:32.984178 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:33.027951 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d60ce-d7b7-4acd-939b-84a3f5365b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:33.028482 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d60ce-d7b7-4acd-939b-84a3f5365b3a admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:33.028794 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d60ce-d7b7-4acd-939b-84a3f5365b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:33.029068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d60ce-d7b7-4acd-939b-84a3f5365b3a admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:33.030430 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d60ce-d7b7-4acd-939b-84a3f5365b3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:33.032320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d60ce-d7b7-4acd-939b-84a3f5365b3a admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:33.102068 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1402/2803] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:32:32 2022] POST /identity/v3/users => generated 309 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:32:33.106203 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a4997-3bed-40f5-a687-88c065682e16 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:33.116701 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a4997-3bed-40f5-a687-88c065682e16 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:33.121610 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9595e8-9c12-46ab-b062-c681a8ac812b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:33.122207 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9595e8-9c12-46ab-b062-c681a8ac812b admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:33.122489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9595e8-9c12-46ab-b062-c681a8ac812b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:33.122763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9595e8-9c12-46ab-b062-c681a8ac812b admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:33.123464 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9595e8-9c12-46ab-b062-c681a8ac812b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:33.125100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9595e8-9c12-46ab-b062-c681a8ac812b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:33.135529 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1402/2804] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:33.140449 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d60ce-d7b7-4acd-939b-84a3f5365b3a admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:33.150198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d60ce-d7b7-4acd-939b-84a3f5365b3a admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:33.155490 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb610a-53fc-4b80-bd73-be2205a7071f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:33.156212 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb610a-53fc-4b80-bd73-be2205a7071f admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:33.156495 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb610a-53fc-4b80-bd73-be2205a7071f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:33.156804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb610a-53fc-4b80-bd73-be2205a7071f admin admin] PATH_INFO: `/v3/projects/f1dab81958dc473087ad0238672d794c/users/a4c139afbac14099a3a495b84cbf2053/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:33.171549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb610a-53fc-4b80-bd73-be2205a7071f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1dab81958dc473087ad0238672d794c, user_id=a4c139afbac14099a3a495b84cbf2053, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:33.174281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb610a-53fc-4b80-bd73-be2205a7071f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:33.231174 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1403/2805] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:32:33 2022] PUT /identity/v3/projects/f1dab81958dc473087ad0238672d794c/users/a4c139afbac14099a3a495b84cbf2053/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:33.235188 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9595e8-9c12-46ab-b062-c681a8ac812b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:33.245311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9595e8-9c12-46ab-b062-c681a8ac812b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:33.303854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdacb17-babe-4eee-a7ff-fd5694362460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:33.304454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdacb17-babe-4eee-a7ff-fd5694362460 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:33.304768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdacb17-babe-4eee-a7ff-fd5694362460 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:33.305051 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdacb17-babe-4eee-a7ff-fd5694362460 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:33.305686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdacb17-babe-4eee-a7ff-fd5694362460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:33.307269 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdacb17-babe-4eee-a7ff-fd5694362460 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:33.316941 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1403/2806] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:33.320845 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb610a-53fc-4b80-bd73-be2205a7071f admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:33.331628 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb610a-53fc-4b80-bd73-be2205a7071f admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:33.336203 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879843ec-ca03-4fa7-bdfd-fe8c35b8c59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:33.336888 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879843ec-ca03-4fa7-bdfd-fe8c35b8c59e admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:33.337174 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879843ec-ca03-4fa7-bdfd-fe8c35b8c59e admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:33.337451 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879843ec-ca03-4fa7-bdfd-fe8c35b8c59e admin admin] PATH_INFO: `/v3/projects/f1dab81958dc473087ad0238672d794c/users/a4c139afbac14099a3a495b84cbf2053/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:33.339361 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879843ec-ca03-4fa7-bdfd-fe8c35b8c59e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1dab81958dc473087ad0238672d794c, user_id=a4c139afbac14099a3a495b84cbf2053, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:33.342675 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879843ec-ca03-4fa7-bdfd-fe8c35b8c59e admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:33.352619 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1404/2807] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:32:33 2022] PUT /identity/v3/projects/f1dab81958dc473087ad0238672d794c/users/a4c139afbac14099a3a495b84cbf2053/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:33.417642 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b5e37-570c-42a4-9712-8ceebdea3feb None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:33.417927 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b5e37-570c-42a4-9712-8ceebdea3feb None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:33.418202 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b5e37-570c-42a4-9712-8ceebdea3feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:33.445060 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-9e7b5e37-570c-42a4-9712-8ceebdea3feb None None] MFA Rules not processed for user `a4c139afbac14099a3a495b84cbf2053`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:33.446262 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9e7b5e37-570c-42a4-9712-8ceebdea3feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:33.512212 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1404/2808] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:33 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:33.524947 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879843ec-ca03-4fa7-bdfd-fe8c35b8c59e admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:33.526752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-879843ec-ca03-4fa7-bdfd-fe8c35b8c59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:33.584269 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879843ec-ca03-4fa7-bdfd-fe8c35b8c59e admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:33.587824 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ccae2-536c-4185-a69a-8b5f3babe10a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:33.588413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ccae2-536c-4185-a69a-8b5f3babe10a service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:33.588722 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ccae2-536c-4185-a69a-8b5f3babe10a service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:33.588998 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ccae2-536c-4185-a69a-8b5f3babe10a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:33.598074 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ccae2-536c-4185-a69a-8b5f3babe10a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:33.599503 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ccae2-536c-4185-a69a-8b5f3babe10a service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:33.608984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1405/2809] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:32:33 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:32:33.743283 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b5e37-570c-42a4-9712-8ceebdea3feb None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:33.760911 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:33.805974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b5e37-570c-42a4-9712-8ceebdea3feb None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:33.811196 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc11d14-04a3-442f-85f6-6237c5e57287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:33.811856 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc11d14-04a3-442f-85f6-6237c5e57287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:33.812132 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc11d14-04a3-442f-85f6-6237c5e57287 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:33.812401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc11d14-04a3-442f-85f6-6237c5e57287 admin admin] PATH_INFO: `/v3/users/47f4b130b63a41629017fa9632472b9a` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:33.813624 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc11d14-04a3-442f-85f6-6237c5e57287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f4b130b63a41629017fa9632472b9a)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:33.815531 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc11d14-04a3-442f-85f6-6237c5e57287 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:33.977194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-dbc11d14-04a3-442f-85f6-6237c5e57287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f4b130b63a41629017fa9632472b9a'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:34.046454 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-dbc11d14-04a3-442f-85f6-6237c5e57287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f4b130b63a41629017fa9632472b9a'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:34.047092 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-dbc11d14-04a3-442f-85f6-6237c5e57287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f4b130b63a41629017fa9632472b9a'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:34.054579 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-dbc11d14-04a3-442f-85f6-6237c5e57287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f4b130b63a41629017fa9632472b9a'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:34.067354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-dbc11d14-04a3-442f-85f6-6237c5e57287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f4b130b63a41629017fa9632472b9a'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:34.073273 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1405/2810] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:32:33 2022] DELETE /identity/v3/users/47f4b130b63a41629017fa9632472b9a => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:34.077832 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5a62f-7e32-4945-a71b-b0411450c33e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:34.078121 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5a62f-7e32-4945-a71b-b0411450c33e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:34.078398 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5a62f-7e32-4945-a71b-b0411450c33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:34.097136 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-85e5a62f-7e32-4945-a71b-b0411450c33e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:34.098543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-85e5a62f-7e32-4945-a71b-b0411450c33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:34.155362 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1406/2811] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:34.165274 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc11d14-04a3-442f-85f6-6237c5e57287 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:34.166723 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-dbc11d14-04a3-442f-85f6-6237c5e57287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:34.225217 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc11d14-04a3-442f-85f6-6237c5e57287 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:34.228978 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4a733-97fd-427f-a28b-ae9e5c4ed117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:34.229533 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4a733-97fd-427f-a28b-ae9e5c4ed117 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:34.229815 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4a733-97fd-427f-a28b-ae9e5c4ed117 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:34.230090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4a733-97fd-427f-a28b-ae9e5c4ed117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:34.239925 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4a733-97fd-427f-a28b-ae9e5c4ed117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:34.241407 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4a733-97fd-427f-a28b-ae9e5c4ed117 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:34.251075 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1406/2812] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:32:34.708416 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5a62f-7e32-4945-a71b-b0411450c33e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:34.709927 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-85e5a62f-7e32-4945-a71b-b0411450c33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:34.723288 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5a62f-7e32-4945-a71b-b0411450c33e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:34.727911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:34.728538 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:34.728979 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:34.729259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] PATH_INFO: `/v3/projects/46d2c48079ed4ab085cccbcedc7bffcb` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:34.730389 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46d2c48079ed4ab085cccbcedc7bffcb)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:34.732293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:34.741214 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:34.844117 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46d2c48079ed4ab085cccbcedc7bffcb', 'request_id': 'req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:34.850316 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46d2c48079ed4ab085cccbcedc7bffcb', 'request_id': 'req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:34.856429 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46d2c48079ed4ab085cccbcedc7bffcb', 'request_id': 'req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:34.922577 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] The token cache is being invalidate because project 46d2c48079ed4ab085cccbcedc7bffcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:32:34.922913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:34.923529 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:34.925044 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1407/2813] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:32:34 2022] DELETE /identity/v3/projects/46d2c48079ed4ab085cccbcedc7bffcb => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:32:34.929009 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4a733-97fd-427f-a28b-ae9e5c4ed117 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:34.938764 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4a733-97fd-427f-a28b-ae9e5c4ed117 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:34.973607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:34.995740 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e4432-b337-4ca8-899c-bae4dde80c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:34.996411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e4432-b337-4ca8-899c-bae4dde80c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:34.996803 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e4432-b337-4ca8-899c-bae4dde80c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:34.997081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e4432-b337-4ca8-899c-bae4dde80c2a admin admin] PATH_INFO: `/v3/users/a4c139afbac14099a3a495b84cbf2053` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:34.998872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e4432-b337-4ca8-899c-bae4dde80c2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4c139afbac14099a3a495b84cbf2053)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:35.001441 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e4432-b337-4ca8-899c-bae4dde80c2a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:35.213208 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1c2e4432-b337-4ca8-899c-bae4dde80c2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4c139afbac14099a3a495b84cbf2053'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:35.312415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1c2e4432-b337-4ca8-899c-bae4dde80c2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4c139afbac14099a3a495b84cbf2053'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:35.313065 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1c2e4432-b337-4ca8-899c-bae4dde80c2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4c139afbac14099a3a495b84cbf2053'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:35.320419 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1c2e4432-b337-4ca8-899c-bae4dde80c2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4c139afbac14099a3a495b84cbf2053'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:35.333017 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1c2e4432-b337-4ca8-899c-bae4dde80c2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4c139afbac14099a3a495b84cbf2053'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:35.338963 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1407/2814] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:32:34 2022] DELETE /identity/v3/users/a4c139afbac14099a3a495b84cbf2053 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:35.741616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:35.743080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:35.757024 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5e087-b838-4a61-af7e-c2b5e0b2c2d0 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:35.814707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a94f95-28f9-442f-824e-267cc74dbae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:35.815380 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a94f95-28f9-442f-824e-267cc74dbae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:35.815687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a94f95-28f9-442f-824e-267cc74dbae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:35.815957 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a94f95-28f9-442f-824e-267cc74dbae0 admin admin] PATH_INFO: `/v3/projects/f1dab81958dc473087ad0238672d794c` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:35.817083 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a94f95-28f9-442f-824e-267cc74dbae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1dab81958dc473087ad0238672d794c)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:35.818978 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a94f95-28f9-442f-824e-267cc74dbae0 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:35.945862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-70a94f95-28f9-442f-824e-267cc74dbae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1dab81958dc473087ad0238672d794c', 'request_id': 'req-70a94f95-28f9-442f-824e-267cc74dbae0'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:35.952632 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-70a94f95-28f9-442f-824e-267cc74dbae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1dab81958dc473087ad0238672d794c', 'request_id': 'req-70a94f95-28f9-442f-824e-267cc74dbae0'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:35.958154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-70a94f95-28f9-442f-824e-267cc74dbae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1dab81958dc473087ad0238672d794c', 'request_id': 'req-70a94f95-28f9-442f-824e-267cc74dbae0'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:36.048352 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-70a94f95-28f9-442f-824e-267cc74dbae0 admin admin] The token cache is being invalidate because project f1dab81958dc473087ad0238672d794c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:32:36.048582 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-70a94f95-28f9-442f-824e-267cc74dbae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:36.049208 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-70a94f95-28f9-442f-824e-267cc74dbae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:36.050739 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1408/2815] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:32:35 2022] DELETE /identity/v3/projects/f1dab81958dc473087ad0238672d794c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:36.055611 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d59c3-f827-4e69-9f7b-783a7f5920a0 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:36.055918 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d59c3-f827-4e69-9f7b-783a7f5920a0 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:36.056194 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d59c3-f827-4e69-9f7b-783a7f5920a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:36.075839 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-b42d59c3-f827-4e69-9f7b-783a7f5920a0 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:36.077704 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-b42d59c3-f827-4e69-9f7b-783a7f5920a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:36.134506 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1408/2816] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:36.141960 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6ae7d-d089-4486-ae6d-66cfe29cea31 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:36.142257 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6ae7d-d089-4486-ae6d-66cfe29cea31 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:36.142542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6ae7d-d089-4486-ae6d-66cfe29cea31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:36.162226 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-c4d6ae7d-d089-4486-ae6d-66cfe29cea31 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:36.163428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-c4d6ae7d-d089-4486-ae6d-66cfe29cea31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:36.171612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1409/2817] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:36.175626 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d59c3-f827-4e69-9f7b-783a7f5920a0 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:36.185061 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d59c3-f827-4e69-9f7b-783a7f5920a0 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:36.190295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5cb90-8f5a-47cc-9fc6-a176df86d55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:36.190873 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5cb90-8f5a-47cc-9fc6-a176df86d55e admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:36.191149 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5cb90-8f5a-47cc-9fc6-a176df86d55e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:36.191415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5cb90-8f5a-47cc-9fc6-a176df86d55e admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:36.192048 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5cb90-8f5a-47cc-9fc6-a176df86d55e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:32:36.192348 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5cb90-8f5a-47cc-9fc6-a176df86d55e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:36.193951 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5cb90-8f5a-47cc-9fc6-a176df86d55e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:36.205733 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1409/2818] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:32:36.209787 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6ae7d-d089-4486-ae6d-66cfe29cea31 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:36.219373 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6ae7d-d089-4486-ae6d-66cfe29cea31 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:36.223965 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164c448-7e74-4614-b2f0-c57d65392505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:36.224551 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164c448-7e74-4614-b2f0-c57d65392505 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:36.224866 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164c448-7e74-4614-b2f0-c57d65392505 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:36.225143 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164c448-7e74-4614-b2f0-c57d65392505 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:36.225770 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164c448-7e74-4614-b2f0-c57d65392505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:36.227648 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164c448-7e74-4614-b2f0-c57d65392505 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:36.309011 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:36.316051 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1410/2819] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:32:36 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:32:36.320865 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5cb90-8f5a-47cc-9fc6-a176df86d55e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:36.331451 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5cb90-8f5a-47cc-9fc6-a176df86d55e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:36.371996 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:36.386117 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1369f2-b4b7-475a-b408-793ba3258c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:36.386712 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1369f2-b4b7-475a-b408-793ba3258c67 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:36.386990 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1369f2-b4b7-475a-b408-793ba3258c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:36.387282 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1369f2-b4b7-475a-b408-793ba3258c67 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:36.387890 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1369f2-b4b7-475a-b408-793ba3258c67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:36.390368 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1369f2-b4b7-475a-b408-793ba3258c67 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:36.469760 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1410/2820] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:32:36 2022] POST /identity/v3/users => generated 283 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:32:36.473782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164c448-7e74-4614-b2f0-c57d65392505 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:36.485580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164c448-7e74-4614-b2f0-c57d65392505 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:36.490356 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82451bf-9691-42ea-b2a6-4c9860ceea28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:36.491038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82451bf-9691-42ea-b2a6-4c9860ceea28 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:36.491324 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82451bf-9691-42ea-b2a6-4c9860ceea28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:36.491624 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82451bf-9691-42ea-b2a6-4c9860ceea28 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:36.492262 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82451bf-9691-42ea-b2a6-4c9860ceea28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:36.493875 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82451bf-9691-42ea-b2a6-4c9860ceea28 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:36.504716 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1411/2821] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:32:36.508604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1369f2-b4b7-475a-b408-793ba3258c67 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:36.519625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1369f2-b4b7-475a-b408-793ba3258c67 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:36.524256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08dd19-10d6-4b48-ba43-efc15ab4f074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:36.524866 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08dd19-10d6-4b48-ba43-efc15ab4f074 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:36.525148 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08dd19-10d6-4b48-ba43-efc15ab4f074 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:36.525421 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08dd19-10d6-4b48-ba43-efc15ab4f074 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:36.526056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08dd19-10d6-4b48-ba43-efc15ab4f074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:36.527692 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08dd19-10d6-4b48-ba43-efc15ab4f074 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:36.539825 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1411/2822] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:36.543890 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82451bf-9691-42ea-b2a6-4c9860ceea28 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:36.553870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82451bf-9691-42ea-b2a6-4c9860ceea28 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:36.558543 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a541ce-3bf6-4c5e-8654-7e17cea730f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:36.559240 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a541ce-3bf6-4c5e-8654-7e17cea730f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:36.559465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a541ce-3bf6-4c5e-8654-7e17cea730f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:36.559741 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a541ce-3bf6-4c5e-8654-7e17cea730f2 admin admin] PATH_INFO: `/v3/projects/af22d2cb09794c79a718fe00e3334946/users/43f04fc09e4a4d3a8664e7f7a6a4b46b/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:36.574694 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a541ce-3bf6-4c5e-8654-7e17cea730f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af22d2cb09794c79a718fe00e3334946, user_id=43f04fc09e4a4d3a8664e7f7a6a4b46b, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:36.578182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a541ce-3bf6-4c5e-8654-7e17cea730f2 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:36.664285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1412/2823] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:32:36 2022] PUT /identity/v3/projects/af22d2cb09794c79a718fe00e3334946/users/43f04fc09e4a4d3a8664e7f7a6a4b46b/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:36.674783 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17861ca-b846-48dd-a679-a5ab81505013 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:36.675067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17861ca-b846-48dd-a679-a5ab81505013 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:36.675347 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17861ca-b846-48dd-a679-a5ab81505013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:36.703846 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e17861ca-b846-48dd-a679-a5ab81505013 None None] MFA Rules not processed for user `43f04fc09e4a4d3a8664e7f7a6a4b46b`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:36.705160 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e17861ca-b846-48dd-a679-a5ab81505013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:36.770005 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1412/2824] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:36.774585 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a541ce-3bf6-4c5e-8654-7e17cea730f2 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:36.838382 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a541ce-3bf6-4c5e-8654-7e17cea730f2 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:36.843541 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd73817-2e01-4e74-9c8b-7aef88734a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:36.844140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd73817-2e01-4e74-9c8b-7aef88734a53 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:36.844422 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd73817-2e01-4e74-9c8b-7aef88734a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:36.844728 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd73817-2e01-4e74-9c8b-7aef88734a53 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:36.845350 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd73817-2e01-4e74-9c8b-7aef88734a53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:36.847287 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd73817-2e01-4e74-9c8b-7aef88734a53 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:36.935740 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1413/2825] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:32:36 2022] POST /identity/v3/projects => generated 367 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:32:36.939795 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17861ca-b846-48dd-a679-a5ab81505013 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:36.950398 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17861ca-b846-48dd-a679-a5ab81505013 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:37.005896 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0704ed-8985-40a6-bda8-c8a7f0ec5b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:37.006492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0704ed-8985-40a6-bda8-c8a7f0ec5b8a admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:37.006775 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0704ed-8985-40a6-bda8-c8a7f0ec5b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:37.007054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0704ed-8985-40a6-bda8-c8a7f0ec5b8a admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:37.007666 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0704ed-8985-40a6-bda8-c8a7f0ec5b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:37.009627 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0704ed-8985-40a6-bda8-c8a7f0ec5b8a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:37.106383 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1413/2826] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:32:36 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:32:37.109710 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd73817-2e01-4e74-9c8b-7aef88734a53 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:37.119782 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd73817-2e01-4e74-9c8b-7aef88734a53 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:37.124701 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286add0-2231-4907-b77b-b6de0518bd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:37.125289 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286add0-2231-4907-b77b-b6de0518bd22 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:37.125567 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286add0-2231-4907-b77b-b6de0518bd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:37.125839 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286add0-2231-4907-b77b-b6de0518bd22 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:37.126468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286add0-2231-4907-b77b-b6de0518bd22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:37.128161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286add0-2231-4907-b77b-b6de0518bd22 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:37.139195 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1414/2827] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:37.143391 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0704ed-8985-40a6-bda8-c8a7f0ec5b8a admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:37.153166 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0704ed-8985-40a6-bda8-c8a7f0ec5b8a admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:37.158455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5234b3-c7e6-410d-a68d-e1fe79fc9aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:37.159090 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5234b3-c7e6-410d-a68d-e1fe79fc9aab admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:37.159372 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5234b3-c7e6-410d-a68d-e1fe79fc9aab admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:37.159654 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5234b3-c7e6-410d-a68d-e1fe79fc9aab admin admin] PATH_INFO: `/v3/projects/c2629b133f3d4cb99d0cf1f1b90c117d/users/d91acc4a54d0404085b8d5a38ee9536b/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:37.174178 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5234b3-c7e6-410d-a68d-e1fe79fc9aab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2629b133f3d4cb99d0cf1f1b90c117d, user_id=d91acc4a54d0404085b8d5a38ee9536b, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:37.176857 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5234b3-c7e6-410d-a68d-e1fe79fc9aab admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:37.243063 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1414/2828] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:32:37 2022] PUT /identity/v3/projects/c2629b133f3d4cb99d0cf1f1b90c117d/users/d91acc4a54d0404085b8d5a38ee9536b/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:37.246941 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286add0-2231-4907-b77b-b6de0518bd22 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:37.257727 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286add0-2231-4907-b77b-b6de0518bd22 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:37.314109 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ca9aa-95e3-49a8-af8e-7b02e1c52a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:37.314707 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ca9aa-95e3-49a8-af8e-7b02e1c52a10 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:37.314990 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ca9aa-95e3-49a8-af8e-7b02e1c52a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:37.315264 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ca9aa-95e3-49a8-af8e-7b02e1c52a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:37.315895 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ca9aa-95e3-49a8-af8e-7b02e1c52a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:37.317507 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ca9aa-95e3-49a8-af8e-7b02e1c52a10 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:37.327078 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1415/2829] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:37.331102 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5234b3-c7e6-410d-a68d-e1fe79fc9aab admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:37.341499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5234b3-c7e6-410d-a68d-e1fe79fc9aab admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:37.346687 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ce781-a6e0-4692-b46e-83fb38cddc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:37.347335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ce781-a6e0-4692-b46e-83fb38cddc0e admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:37.347618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ce781-a6e0-4692-b46e-83fb38cddc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:37.347898 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ce781-a6e0-4692-b46e-83fb38cddc0e admin admin] PATH_INFO: `/v3/projects/c2629b133f3d4cb99d0cf1f1b90c117d/users/d91acc4a54d0404085b8d5a38ee9536b/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:37.349842 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ce781-a6e0-4692-b46e-83fb38cddc0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2629b133f3d4cb99d0cf1f1b90c117d, user_id=d91acc4a54d0404085b8d5a38ee9536b, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:37.352480 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ce781-a6e0-4692-b46e-83fb38cddc0e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:37.363267 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1415/2830] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:32:37 2022] PUT /identity/v3/projects/c2629b133f3d4cb99d0cf1f1b90c117d/users/d91acc4a54d0404085b8d5a38ee9536b/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:37.374590 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbda54-a9ba-4f5f-a4d5-948c8b2fe8f7 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:37.374880 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbda54-a9ba-4f5f-a4d5-948c8b2fe8f7 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:37.375157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbda54-a9ba-4f5f-a4d5-948c8b2fe8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:37.402428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-6ccbda54-a9ba-4f5f-a4d5-948c8b2fe8f7 None None] MFA Rules not processed for user `d91acc4a54d0404085b8d5a38ee9536b`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:37.403676 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6ccbda54-a9ba-4f5f-a4d5-948c8b2fe8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:37.423306 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:37.470920 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1416/2831] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:37.482165 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ce781-a6e0-4692-b46e-83fb38cddc0e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:37.483988 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-850ce781-a6e0-4692-b46e-83fb38cddc0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:37.490501 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:37.542467 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ce781-a6e0-4692-b46e-83fb38cddc0e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:37.546598 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6044619-3896-4972-9ab5-63df9fcbfc9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:37.547206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044619-3896-4972-9ab5-63df9fcbfc9b service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:37.547903 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044619-3896-4972-9ab5-63df9fcbfc9b service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:37.548176 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044619-3896-4972-9ab5-63df9fcbfc9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:37.557062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6044619-3896-4972-9ab5-63df9fcbfc9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:37.558492 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6044619-3896-4972-9ab5-63df9fcbfc9b service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:37.568595 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1416/2832] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:32:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:32:37.829005 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbda54-a9ba-4f5f-a4d5-948c8b2fe8f7 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:37.838691 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbda54-a9ba-4f5f-a4d5-948c8b2fe8f7 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:37.842485 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5f3b0-06a4-4a0d-a563-5fe32ec0e6fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:37.843081 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5f3b0-06a4-4a0d-a563-5fe32ec0e6fc service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:37.843438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5f3b0-06a4-4a0d-a563-5fe32ec0e6fc service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:37.843709 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5f3b0-06a4-4a0d-a563-5fe32ec0e6fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:37.898531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5f3b0-06a4-4a0d-a563-5fe32ec0e6fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:37.899964 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5f3b0-06a4-4a0d-a563-5fe32ec0e6fc service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:37.908537 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1417/2833] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:32:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:32:44.340503 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6044619-3896-4972-9ab5-63df9fcbfc9b service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:44.401995 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6044619-3896-4972-9ab5-63df9fcbfc9b service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:44.407284 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c60e8-ab1d-4785-9f1e-f93b62f7863c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:44.407974 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c60e8-ab1d-4785-9f1e-f93b62f7863c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:44.408255 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c60e8-ab1d-4785-9f1e-f93b62f7863c admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:44.408537 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c60e8-ab1d-4785-9f1e-f93b62f7863c admin admin] PATH_INFO: `/v3/users/43f04fc09e4a4d3a8664e7f7a6a4b46b` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:44.409720 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c60e8-ab1d-4785-9f1e-f93b62f7863c admin admin] RBAC: Authorizing `identity:delete_user(user_id=43f04fc09e4a4d3a8664e7f7a6a4b46b)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:44.411906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c60e8-ab1d-4785-9f1e-f93b62f7863c admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:44.564932 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-9b8c60e8-ab1d-4785-9f1e-f93b62f7863c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43f04fc09e4a4d3a8664e7f7a6a4b46b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:44.631332 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-9b8c60e8-ab1d-4785-9f1e-f93b62f7863c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43f04fc09e4a4d3a8664e7f7a6a4b46b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:44.632016 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-9b8c60e8-ab1d-4785-9f1e-f93b62f7863c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43f04fc09e4a4d3a8664e7f7a6a4b46b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:44.639682 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-9b8c60e8-ab1d-4785-9f1e-f93b62f7863c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43f04fc09e4a4d3a8664e7f7a6a4b46b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:44.651818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-9b8c60e8-ab1d-4785-9f1e-f93b62f7863c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43f04fc09e4a4d3a8664e7f7a6a4b46b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:44.658214 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1417/2834] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:32:44 2022] DELETE /identity/v3/users/43f04fc09e4a4d3a8664e7f7a6a4b46b => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:44.663054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9630-eeac-44c0-9b05-aa365207168e None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:44.663531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9630-eeac-44c0-9b05-aa365207168e None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:44.663637 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9630-eeac-44c0-9b05-aa365207168e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:44.682428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-f9fb9630-eeac-44c0-9b05-aa365207168e None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:44.683619 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f9fb9630-eeac-44c0-9b05-aa365207168e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:44.740843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1418/2835] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:44.754354 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c60e8-ab1d-4785-9f1e-f93b62f7863c admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:44.755747 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-9b8c60e8-ab1d-4785-9f1e-f93b62f7863c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:44.786620 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:44.811893 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c60e8-ab1d-4785-9f1e-f93b62f7863c admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:44.816310 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b2a73-2d3e-4bda-ba17-2d2950299403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:44.817068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b2a73-2d3e-4bda-ba17-2d2950299403 service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:44.817773 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b2a73-2d3e-4bda-ba17-2d2950299403 service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:44.818028 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b2a73-2d3e-4bda-ba17-2d2950299403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:44.826680 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b2a73-2d3e-4bda-ba17-2d2950299403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:44.828710 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b2a73-2d3e-4bda-ba17-2d2950299403 service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:44.837834 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1418/2836] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:32:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:32:45.253549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb9630-eeac-44c0-9b05-aa365207168e None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:45.255010 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f9fb9630-eeac-44c0-9b05-aa365207168e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:45.268771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb9630-eeac-44c0-9b05-aa365207168e None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:45.273466 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:45.274099 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:45.274380 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:45.274656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] PATH_INFO: `/v3/projects/af22d2cb09794c79a718fe00e3334946` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:45.276760 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af22d2cb09794c79a718fe00e3334946)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:45.278683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:45.371018 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af22d2cb09794c79a718fe00e3334946', 'request_id': 'req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:45.372903 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:45.378006 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af22d2cb09794c79a718fe00e3334946', 'request_id': 'req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:45.383475 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af22d2cb09794c79a718fe00e3334946', 'request_id': 'req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:45.449946 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] The token cache is being invalidate because project af22d2cb09794c79a718fe00e3334946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:32:45.450235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:45.450861 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:45.452340 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1419/2837] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:32:45 2022] DELETE /identity/v3/projects/af22d2cb09794c79a718fe00e3334946 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:32:45.456420 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b2a73-2d3e-4bda-ba17-2d2950299403 service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:45.465947 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b2a73-2d3e-4bda-ba17-2d2950299403 service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:45.522283 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bf1be-499d-4efb-bfa7-fad1718f5cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:45.522823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bf1be-499d-4efb-bfa7-fad1718f5cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:45.523100 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bf1be-499d-4efb-bfa7-fad1718f5cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:45.523369 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bf1be-499d-4efb-bfa7-fad1718f5cc7 admin admin] PATH_INFO: `/v3/users/d91acc4a54d0404085b8d5a38ee9536b` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:45.524523 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bf1be-499d-4efb-bfa7-fad1718f5cc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d91acc4a54d0404085b8d5a38ee9536b)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:45.526494 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bf1be-499d-4efb-bfa7-fad1718f5cc7 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:45.754301 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4e6bf1be-499d-4efb-bfa7-fad1718f5cc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd91acc4a54d0404085b8d5a38ee9536b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:45.825438 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4e6bf1be-499d-4efb-bfa7-fad1718f5cc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd91acc4a54d0404085b8d5a38ee9536b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:45.826071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4e6bf1be-499d-4efb-bfa7-fad1718f5cc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd91acc4a54d0404085b8d5a38ee9536b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:45.834180 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4e6bf1be-499d-4efb-bfa7-fad1718f5cc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd91acc4a54d0404085b8d5a38ee9536b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:45.846794 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-4e6bf1be-499d-4efb-bfa7-fad1718f5cc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd91acc4a54d0404085b8d5a38ee9536b'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:45.852625 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1419/2838] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:32:45 2022] DELETE /identity/v3/users/d91acc4a54d0404085b8d5a38ee9536b => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:46.232771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:46.234259 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:46.251164 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc44d90-9b9d-43ce-bd99-37bdb7d36e79 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:46.311592 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43099528-a4a3-47ff-9611-5988bbd2bf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:46.312213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43099528-a4a3-47ff-9611-5988bbd2bf12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:46.313180 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43099528-a4a3-47ff-9611-5988bbd2bf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:46.313474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43099528-a4a3-47ff-9611-5988bbd2bf12 admin admin] PATH_INFO: `/v3/projects/c2629b133f3d4cb99d0cf1f1b90c117d` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:46.314984 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43099528-a4a3-47ff-9611-5988bbd2bf12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2629b133f3d4cb99d0cf1f1b90c117d)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:46.317069 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43099528-a4a3-47ff-9611-5988bbd2bf12 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:46.421981 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-43099528-a4a3-47ff-9611-5988bbd2bf12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2629b133f3d4cb99d0cf1f1b90c117d', 'request_id': 'req-43099528-a4a3-47ff-9611-5988bbd2bf12'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:46.429042 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-43099528-a4a3-47ff-9611-5988bbd2bf12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2629b133f3d4cb99d0cf1f1b90c117d', 'request_id': 'req-43099528-a4a3-47ff-9611-5988bbd2bf12'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:46.435460 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-43099528-a4a3-47ff-9611-5988bbd2bf12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2629b133f3d4cb99d0cf1f1b90c117d', 'request_id': 'req-43099528-a4a3-47ff-9611-5988bbd2bf12'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:46.555206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-43099528-a4a3-47ff-9611-5988bbd2bf12 admin admin] The token cache is being invalidate because project c2629b133f3d4cb99d0cf1f1b90c117d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:32:46.555498 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-43099528-a4a3-47ff-9611-5988bbd2bf12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:46.556199 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-43099528-a4a3-47ff-9611-5988bbd2bf12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:46.557852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1420/2839] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:32:46 2022] DELETE /identity/v3/projects/c2629b133f3d4cb99d0cf1f1b90c117d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:46.563411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b59a60-d52d-41b4-beb3-0e4e914220b2 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:46.563874 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b59a60-d52d-41b4-beb3-0e4e914220b2 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:46.563982 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b59a60-d52d-41b4-beb3-0e4e914220b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:46.582980 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-01b59a60-d52d-41b4-beb3-0e4e914220b2 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:46.584172 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-01b59a60-d52d-41b4-beb3-0e4e914220b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:46.630895 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:46.643918 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1420/2840] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:46.651500 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1764f-e42f-47c6-90d5-a95dd9d4b46b None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:46.651794 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1764f-e42f-47c6-90d5-a95dd9d4b46b None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:46.652077 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1764f-e42f-47c6-90d5-a95dd9d4b46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:46.672856 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-9fd1764f-e42f-47c6-90d5-a95dd9d4b46b None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:46.673997 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-9fd1764f-e42f-47c6-90d5-a95dd9d4b46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:46.681868 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1421/2841] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:46.686139 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b59a60-d52d-41b4-beb3-0e4e914220b2 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:46.696366 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b59a60-d52d-41b4-beb3-0e4e914220b2 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:46.700960 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1e07a-87b5-456b-868a-3e8856482938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:46.701564 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1e07a-87b5-456b-868a-3e8856482938 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:46.701845 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1e07a-87b5-456b-868a-3e8856482938 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:46.702183 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1e07a-87b5-456b-868a-3e8856482938 admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:46.702863 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1e07a-87b5-456b-868a-3e8856482938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:32:46.703392 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1e07a-87b5-456b-868a-3e8856482938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:46.705025 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1e07a-87b5-456b-868a-3e8856482938 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:46.716698 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1421/2842] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:32:46.721140 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1764f-e42f-47c6-90d5-a95dd9d4b46b None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:46.730447 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1764f-e42f-47c6-90d5-a95dd9d4b46b None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:46.735943 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae857e-5b45-48fd-81ab-d12653c1664a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:46.736540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae857e-5b45-48fd-81ab-d12653c1664a admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:46.736862 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae857e-5b45-48fd-81ab-d12653c1664a admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:46.737141 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae857e-5b45-48fd-81ab-d12653c1664a admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:46.737763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae857e-5b45-48fd-81ab-d12653c1664a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:46.739665 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae857e-5b45-48fd-81ab-d12653c1664a admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:46.805985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1422/2843] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:32:46 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:32:46.810215 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1e07a-87b5-456b-868a-3e8856482938 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:46.821066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1e07a-87b5-456b-868a-3e8856482938 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:46.876555 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3ecf8-eeca-4358-a68f-9966c707603f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:46.877175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3ecf8-eeca-4358-a68f-9966c707603f admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:46.877456 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3ecf8-eeca-4358-a68f-9966c707603f admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:46.877727 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3ecf8-eeca-4358-a68f-9966c707603f admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:46.878338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3ecf8-eeca-4358-a68f-9966c707603f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:46.880241 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3ecf8-eeca-4358-a68f-9966c707603f admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:46.942485 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1422/2844] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:32:46 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:32:46.946629 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae857e-5b45-48fd-81ab-d12653c1664a admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:46.957302 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae857e-5b45-48fd-81ab-d12653c1664a admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:46.961899 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c4736-20f2-420d-b963-7c7d15ed3522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:46.962491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c4736-20f2-420d-b963-7c7d15ed3522 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:46.962774 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c4736-20f2-420d-b963-7c7d15ed3522 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:46.963049 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c4736-20f2-420d-b963-7c7d15ed3522 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:46.963692 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c4736-20f2-420d-b963-7c7d15ed3522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:46.965318 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c4736-20f2-420d-b963-7c7d15ed3522 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:46.975656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1423/2845] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:46.979472 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3ecf8-eeca-4358-a68f-9966c707603f admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:46.989997 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3ecf8-eeca-4358-a68f-9966c707603f admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:46.994622 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b242a-5cc9-46b8-ad1d-5f540246e7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:46.995213 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b242a-5cc9-46b8-ad1d-5f540246e7bd admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:46.995499 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b242a-5cc9-46b8-ad1d-5f540246e7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:46.995775 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b242a-5cc9-46b8-ad1d-5f540246e7bd admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:46.996415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b242a-5cc9-46b8-ad1d-5f540246e7bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:46.998024 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b242a-5cc9-46b8-ad1d-5f540246e7bd admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:47.008468 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1423/2846] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:47.012413 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c4736-20f2-420d-b963-7c7d15ed3522 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:47.022531 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c4736-20f2-420d-b963-7c7d15ed3522 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:47.027061 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b6cb8-8852-4f1b-b41a-1c5fe0867392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:47.027683 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b6cb8-8852-4f1b-b41a-1c5fe0867392 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:47.027970 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b6cb8-8852-4f1b-b41a-1c5fe0867392 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:47.028238 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b6cb8-8852-4f1b-b41a-1c5fe0867392 admin admin] PATH_INFO: `/v3/projects/07736777e4154061947c0fd9763436ec/users/ba8d88e62add4a5e97fb2fd98a124d2e/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:47.043077 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b6cb8-8852-4f1b-b41a-1c5fe0867392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07736777e4154061947c0fd9763436ec, user_id=ba8d88e62add4a5e97fb2fd98a124d2e, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:47.045887 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b6cb8-8852-4f1b-b41a-1c5fe0867392 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:47.118993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1424/2847] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:32:47 2022] PUT /identity/v3/projects/07736777e4154061947c0fd9763436ec/users/ba8d88e62add4a5e97fb2fd98a124d2e/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:47.130580 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1a808-9be4-4807-9f28-a9dba7ca46ef None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:47.131686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1a808-9be4-4807-9f28-a9dba7ca46ef None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:47.131686 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1a808-9be4-4807-9f28-a9dba7ca46ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:47.159120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-4ba1a808-9be4-4807-9f28-a9dba7ca46ef None None] MFA Rules not processed for user `ba8d88e62add4a5e97fb2fd98a124d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:47.160326 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4ba1a808-9be4-4807-9f28-a9dba7ca46ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:47.224467 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1424/2848] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:47.236535 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b6cb8-8852-4f1b-b41a-1c5fe0867392 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:47.238287 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-e82b6cb8-8852-4f1b-b41a-1c5fe0867392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:47.247860 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:47.295183 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b6cb8-8852-4f1b-b41a-1c5fe0867392 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:47.298948 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edbb83-0cdc-4a9d-8731-0d98627cc334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:47.299540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edbb83-0cdc-4a9d-8731-0d98627cc334 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:47.299819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edbb83-0cdc-4a9d-8731-0d98627cc334 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:47.300090 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edbb83-0cdc-4a9d-8731-0d98627cc334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:47.308428 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edbb83-0cdc-4a9d-8731-0d98627cc334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:47.309876 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edbb83-0cdc-4a9d-8731-0d98627cc334 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:47.318474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1425/2849] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:32:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:32:47.340259 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1a808-9be4-4807-9f28-a9dba7ca46ef None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:47.384972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:47.402327 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1a808-9be4-4807-9f28-a9dba7ca46ef None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:47.407838 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2518de-a158-4e52-97d9-5a0fa0dce2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:47.408455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2518de-a158-4e52-97d9-5a0fa0dce2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:47.408781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2518de-a158-4e52-97d9-5a0fa0dce2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:47.409054 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2518de-a158-4e52-97d9-5a0fa0dce2cb admin admin] PATH_INFO: `/v3/users/ba8d88e62add4a5e97fb2fd98a124d2e` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:47.410175 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2518de-a158-4e52-97d9-5a0fa0dce2cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba8d88e62add4a5e97fb2fd98a124d2e)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:47.412333 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2518de-a158-4e52-97d9-5a0fa0dce2cb admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:47.624066 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-bc2518de-a158-4e52-97d9-5a0fa0dce2cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba8d88e62add4a5e97fb2fd98a124d2e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:47.699559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-bc2518de-a158-4e52-97d9-5a0fa0dce2cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba8d88e62add4a5e97fb2fd98a124d2e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:47.700190 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-bc2518de-a158-4e52-97d9-5a0fa0dce2cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba8d88e62add4a5e97fb2fd98a124d2e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:47.708043 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-bc2518de-a158-4e52-97d9-5a0fa0dce2cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba8d88e62add4a5e97fb2fd98a124d2e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:47.719812 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-bc2518de-a158-4e52-97d9-5a0fa0dce2cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba8d88e62add4a5e97fb2fd98a124d2e'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:47.725633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1425/2850] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:32:47 2022] DELETE /identity/v3/users/ba8d88e62add4a5e97fb2fd98a124d2e => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:47.730200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e48cd-5ce6-4959-89f6-92cdb6fded46 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:47.730484 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e48cd-5ce6-4959-89f6-92cdb6fded46 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:47.730788 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e48cd-5ce6-4959-89f6-92cdb6fded46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:47.749687 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-6c3e48cd-5ce6-4959-89f6-92cdb6fded46 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:47.751545 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6c3e48cd-5ce6-4959-89f6-92cdb6fded46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:47.807608 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1426/2851] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:47.818064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2518de-a158-4e52-97d9-5a0fa0dce2cb admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:47.819515 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-bc2518de-a158-4e52-97d9-5a0fa0dce2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:47.877338 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2518de-a158-4e52-97d9-5a0fa0dce2cb admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:47.880971 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b51913-2dca-42bc-99cf-0aea85d1e82a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:47.881569 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b51913-2dca-42bc-99cf-0aea85d1e82a service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:47.881851 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b51913-2dca-42bc-99cf-0aea85d1e82a service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:47.882128 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b51913-2dca-42bc-99cf-0aea85d1e82a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:47.891821 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b51913-2dca-42bc-99cf-0aea85d1e82a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:47.893295 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b51913-2dca-42bc-99cf-0aea85d1e82a service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:47.902530 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1426/2852] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:32:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:32:48.334896 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e48cd-5ce6-4959-89f6-92cdb6fded46 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:48.336882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-6c3e48cd-5ce6-4959-89f6-92cdb6fded46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:48.350325 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e48cd-5ce6-4959-89f6-92cdb6fded46 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:48.354995 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43908536-86a5-4e22-a0bb-24ea1d25fbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:48.355599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43908536-86a5-4e22-a0bb-24ea1d25fbeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:48.355882 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43908536-86a5-4e22-a0bb-24ea1d25fbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:48.356161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43908536-86a5-4e22-a0bb-24ea1d25fbeb admin admin] PATH_INFO: `/v3/projects/07736777e4154061947c0fd9763436ec` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:48.357315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43908536-86a5-4e22-a0bb-24ea1d25fbeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=07736777e4154061947c0fd9763436ec)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:48.359315 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43908536-86a5-4e22-a0bb-24ea1d25fbeb admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:48.477667 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-43908536-86a5-4e22-a0bb-24ea1d25fbeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07736777e4154061947c0fd9763436ec', 'request_id': 'req-43908536-86a5-4e22-a0bb-24ea1d25fbeb'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:48.483971 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-43908536-86a5-4e22-a0bb-24ea1d25fbeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07736777e4154061947c0fd9763436ec', 'request_id': 'req-43908536-86a5-4e22-a0bb-24ea1d25fbeb'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:48.489515 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-43908536-86a5-4e22-a0bb-24ea1d25fbeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07736777e4154061947c0fd9763436ec', 'request_id': 'req-43908536-86a5-4e22-a0bb-24ea1d25fbeb'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:48.587650 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-43908536-86a5-4e22-a0bb-24ea1d25fbeb admin admin] The token cache is being invalidate because project 07736777e4154061947c0fd9763436ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:32:48.587940 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-43908536-86a5-4e22-a0bb-24ea1d25fbeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:48.588558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-43908536-86a5-4e22-a0bb-24ea1d25fbeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:48.590200 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1427/2853] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:32:48 2022] DELETE /identity/v3/projects/07736777e4154061947c0fd9763436ec => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:48.595124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad85a7-4699-4eff-b0ad-fb132810e889 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:48.595461 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad85a7-4699-4eff-b0ad-fb132810e889 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:48.595690 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad85a7-4699-4eff-b0ad-fb132810e889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:48.614840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-26ad85a7-4699-4eff-b0ad-fb132810e889 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:48.616039 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-26ad85a7-4699-4eff-b0ad-fb132810e889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:48.675015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1427/2854] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:48.682714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f16e2-8fb4-4c70-baef-4bac722245a4 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:48.683007 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f16e2-8fb4-4c70-baef-4bac722245a4 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:48.683358 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f16e2-8fb4-4c70-baef-4bac722245a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:48.704015 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-fe2f16e2-8fb4-4c70-baef-4bac722245a4 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:48.705331 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-fe2f16e2-8fb4-4c70-baef-4bac722245a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:48.713550 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1428/2855] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:48.717743 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad85a7-4699-4eff-b0ad-fb132810e889 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:48.727489 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad85a7-4699-4eff-b0ad-fb132810e889 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:48.732089 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd459d9-98ec-4bea-ba51-958453d1cc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:48.732709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd459d9-98ec-4bea-ba51-958453d1cc1d admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:48.732999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd459d9-98ec-4bea-ba51-958453d1cc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:48.733274 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd459d9-98ec-4bea-ba51-958453d1cc1d admin admin] PATH_INFO: `/v3/domains` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:48.733933 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd459d9-98ec-4bea-ba51-958453d1cc1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:32:48.734400 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd459d9-98ec-4bea-ba51-958453d1cc1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:48.735980 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd459d9-98ec-4bea-ba51-958453d1cc1d admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:48.737452 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:48.747691 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1428/2856] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:32:48.753524 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f16e2-8fb4-4c70-baef-4bac722245a4 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:48.762774 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f16e2-8fb4-4c70-baef-4bac722245a4 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:48.768154 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598ef85-11cf-455f-bfaf-86c670d890c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:48.768781 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ef85-11cf-455f-bfaf-86c670d890c6 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:48.769065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ef85-11cf-455f-bfaf-86c670d890c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:48.769367 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ef85-11cf-455f-bfaf-86c670d890c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:48.769987 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598ef85-11cf-455f-bfaf-86c670d890c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:48.771874 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598ef85-11cf-455f-bfaf-86c670d890c6 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:48.867751 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1429/2857] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:32:48 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:32:48.871618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd459d9-98ec-4bea-ba51-958453d1cc1d admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:48.882109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd459d9-98ec-4bea-ba51-958453d1cc1d admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:48.937972 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e589c-ccfa-4287-8d2e-51a7e9e3c964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:48.938568 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e589c-ccfa-4287-8d2e-51a7e9e3c964 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:48.938849 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e589c-ccfa-4287-8d2e-51a7e9e3c964 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:48.939120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e589c-ccfa-4287-8d2e-51a7e9e3c964 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:48.939741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e589c-ccfa-4287-8d2e-51a7e9e3c964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:48.941695 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e589c-ccfa-4287-8d2e-51a7e9e3c964 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:49.003427 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1429/2858] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:32:48 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:32:49.007336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598ef85-11cf-455f-bfaf-86c670d890c6 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:49.014131 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:49.018105 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598ef85-11cf-455f-bfaf-86c670d890c6 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:49.022714 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81268a6e-bcd4-4f5d-94c9-c77189e9c67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:49.023300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81268a6e-bcd4-4f5d-94c9-c77189e9c67d admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:49.023579 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81268a6e-bcd4-4f5d-94c9-c77189e9c67d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:49.023850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81268a6e-bcd4-4f5d-94c9-c77189e9c67d admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:49.024583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81268a6e-bcd4-4f5d-94c9-c77189e9c67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:49.026193 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81268a6e-bcd4-4f5d-94c9-c77189e9c67d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:49.038446 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1430/2859] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:49.044768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e589c-ccfa-4287-8d2e-51a7e9e3c964 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:49.055803 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e589c-ccfa-4287-8d2e-51a7e9e3c964 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:49.060480 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29c216-2b9e-46df-b834-9419744d1eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:49.061131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29c216-2b9e-46df-b834-9419744d1eca admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:49.064532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29c216-2b9e-46df-b834-9419744d1eca admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:49.064532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29c216-2b9e-46df-b834-9419744d1eca admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:49.064532 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f29c216-2b9e-46df-b834-9419744d1eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:49.065223 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f29c216-2b9e-46df-b834-9419744d1eca admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:49.075724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1430/2860] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:49.079974 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81268a6e-bcd4-4f5d-94c9-c77189e9c67d admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:49.089763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81268a6e-bcd4-4f5d-94c9-c77189e9c67d admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:49.094769 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b4b5f-20c5-44a9-9c2e-53d80112d0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:49.095409 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b4b5f-20c5-44a9-9c2e-53d80112d0df admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:49.095690 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b4b5f-20c5-44a9-9c2e-53d80112d0df admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:49.095963 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b4b5f-20c5-44a9-9c2e-53d80112d0df admin admin] PATH_INFO: `/v3/projects/e9d7796441824ab997b675a4ba02663e/users/cc917d8dac4248a586dd9fddea086580/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:49.111719 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b4b5f-20c5-44a9-9c2e-53d80112d0df admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9d7796441824ab997b675a4ba02663e, user_id=cc917d8dac4248a586dd9fddea086580, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:49.114423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b4b5f-20c5-44a9-9c2e-53d80112d0df admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:49.196759 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1431/2861] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:32:49 2022] PUT /identity/v3/projects/e9d7796441824ab997b675a4ba02663e/users/cc917d8dac4248a586dd9fddea086580/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:49.207680 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050774c-6f1e-4c67-a7ce-0ebf771f580c None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:49.207999 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050774c-6f1e-4c67-a7ce-0ebf771f580c None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:49.208309 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050774c-6f1e-4c67-a7ce-0ebf771f580c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:49.235465 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-0050774c-6f1e-4c67-a7ce-0ebf771f580c None None] MFA Rules not processed for user `cc917d8dac4248a586dd9fddea086580`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:49.236711 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0050774c-6f1e-4c67-a7ce-0ebf771f580c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:49.300256 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1431/2862] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:49.304703 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b4b5f-20c5-44a9-9c2e-53d80112d0df admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:49.364143 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b4b5f-20c5-44a9-9c2e-53d80112d0df admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:49.368705 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1666157-0720-48f2-95f5-98276adfebe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:49.369298 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1666157-0720-48f2-95f5-98276adfebe3 admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:49.369578 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1666157-0720-48f2-95f5-98276adfebe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:49.369850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1666157-0720-48f2-95f5-98276adfebe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:49.370459 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1666157-0720-48f2-95f5-98276adfebe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:49.372344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1666157-0720-48f2-95f5-98276adfebe3 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:49.469249 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1432/2863] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:32:49 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:32:49.473255 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050774c-6f1e-4c67-a7ce-0ebf771f580c None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:49.483809 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050774c-6f1e-4c67-a7ce-0ebf771f580c None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:49.540525 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819925d-86a0-49ec-8303-b5e6b344475a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:49.541163 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819925d-86a0-49ec-8303-b5e6b344475a admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:49.541445 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819925d-86a0-49ec-8303-b5e6b344475a admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:49.541768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819925d-86a0-49ec-8303-b5e6b344475a admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:49.542403 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819925d-86a0-49ec-8303-b5e6b344475a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:49.544315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819925d-86a0-49ec-8303-b5e6b344475a admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:49.553880 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:49.645544 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1432/2864] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:32:49 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:32:49.649610 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1666157-0720-48f2-95f5-98276adfebe3 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:49.660281 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1666157-0720-48f2-95f5-98276adfebe3 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:49.665204 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f5a20-c277-4c7f-8ba6-3529d02d58b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:49.665793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f5a20-c277-4c7f-8ba6-3529d02d58b3 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:49.666223 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f5a20-c277-4c7f-8ba6-3529d02d58b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:49.666501 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f5a20-c277-4c7f-8ba6-3529d02d58b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:49.667165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f5a20-c277-4c7f-8ba6-3529d02d58b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:49.668793 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f5a20-c277-4c7f-8ba6-3529d02d58b3 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:49.681749 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1433/2865] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:49.685740 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819925d-86a0-49ec-8303-b5e6b344475a admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:49.696768 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819925d-86a0-49ec-8303-b5e6b344475a admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:49.701363 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3704f2-58f4-4206-b06f-f110be28d9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:49.701955 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3704f2-58f4-4206-b06f-f110be28d9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:49.702239 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3704f2-58f4-4206-b06f-f110be28d9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:49.702514 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3704f2-58f4-4206-b06f-f110be28d9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:49.703395 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3704f2-58f4-4206-b06f-f110be28d9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:49.704980 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3704f2-58f4-4206-b06f-f110be28d9f8 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:49.714731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1433/2866] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:49.718695 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f5a20-c277-4c7f-8ba6-3529d02d58b3 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:49.728797 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f5a20-c277-4c7f-8ba6-3529d02d58b3 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:49.733344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28814784-2c52-4b40-b1e1-63dcbcd57602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:49.733993 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28814784-2c52-4b40-b1e1-63dcbcd57602 admin admin] REQUEST_METHOD: `PUT` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:49.734764 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28814784-2c52-4b40-b1e1-63dcbcd57602 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:49.735035 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28814784-2c52-4b40-b1e1-63dcbcd57602 admin admin] PATH_INFO: `/v3/projects/5908ad9b12e94291b50626adec171db1/users/49a0f86c9f424d8e9f13362170e65eed/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:49.749431 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28814784-2c52-4b40-b1e1-63dcbcd57602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5908ad9b12e94291b50626adec171db1, user_id=49a0f86c9f424d8e9f13362170e65eed, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:49.752976 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28814784-2c52-4b40-b1e1-63dcbcd57602 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:49.819704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1434/2867] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:32:49 2022] PUT /identity/v3/projects/5908ad9b12e94291b50626adec171db1/users/49a0f86c9f424d8e9f13362170e65eed/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:49.831023 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011e83e-9e1b-4aca-917a-d321844b8aee None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:49.831311 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011e83e-9e1b-4aca-917a-d321844b8aee None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:49.831590 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011e83e-9e1b-4aca-917a-d321844b8aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:49.858389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-e011e83e-9e1b-4aca-917a-d321844b8aee None None] MFA Rules not processed for user `49a0f86c9f424d8e9f13362170e65eed`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:49.860116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-e011e83e-9e1b-4aca-917a-d321844b8aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:49.923760 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1434/2868] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:49.928198 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28814784-2c52-4b40-b1e1-63dcbcd57602 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:49.989491 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28814784-2c52-4b40-b1e1-63dcbcd57602 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:49.993996 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df3dd9-e956-4c46-94f6-69ef4f56edff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:49.994584 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df3dd9-e956-4c46-94f6-69ef4f56edff admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:49.994883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df3dd9-e956-4c46-94f6-69ef4f56edff admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:49.995135 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df3dd9-e956-4c46-94f6-69ef4f56edff admin admin] PATH_INFO: `/v3/projects` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:49.995741 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df3dd9-e956-4c46-94f6-69ef4f56edff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:49.997655 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df3dd9-e956-4c46-94f6-69ef4f56edff admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:50.124636 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1435/2869] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:32:49 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:32:50.128786 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011e83e-9e1b-4aca-917a-d321844b8aee None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:50.139281 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011e83e-9e1b-4aca-917a-d321844b8aee None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:50.196099 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749bd71-e508-41d1-91fe-67b98e7b90f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:50.196717 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749bd71-e508-41d1-91fe-67b98e7b90f7 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:50.197008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749bd71-e508-41d1-91fe-67b98e7b90f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:50.197283 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749bd71-e508-41d1-91fe-67b98e7b90f7 admin admin] PATH_INFO: `/v3/users` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:50.197907 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749bd71-e508-41d1-91fe-67b98e7b90f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:50.199811 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749bd71-e508-41d1-91fe-67b98e7b90f7 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:50.300313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1435/2870] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:32:50 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:32:50.304335 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df3dd9-e956-4c46-94f6-69ef4f56edff admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:50.311062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:50.315615 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df3dd9-e956-4c46-94f6-69ef4f56edff admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:50.320213 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee74d5d-6e19-4dea-b885-9b4fcb9050d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:50.320844 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee74d5d-6e19-4dea-b885-9b4fcb9050d8 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:50.321147 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee74d5d-6e19-4dea-b885-9b4fcb9050d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:50.321404 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee74d5d-6e19-4dea-b885-9b4fcb9050d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:50.322040 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee74d5d-6e19-4dea-b885-9b4fcb9050d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:50.323628 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee74d5d-6e19-4dea-b885-9b4fcb9050d8 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:50.334985 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1436/2871] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:50.339343 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749bd71-e508-41d1-91fe-67b98e7b90f7 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:50.349835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749bd71-e508-41d1-91fe-67b98e7b90f7 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:50.354467 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d2d05-eef3-456b-8734-28519f02154e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:50.355109 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d2d05-eef3-456b-8734-28519f02154e admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:50.355390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d2d05-eef3-456b-8734-28519f02154e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:50.355663 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d2d05-eef3-456b-8734-28519f02154e admin admin] PATH_INFO: `/v3/projects/d57d10b1337b46afb9c18b949a0f5f3f/users/f15385ec62d34b8fac86a44874e75e8c/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:50.370851 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d2d05-eef3-456b-8734-28519f02154e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57d10b1337b46afb9c18b949a0f5f3f, user_id=f15385ec62d34b8fac86a44874e75e8c, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:50.373633 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d2d05-eef3-456b-8734-28519f02154e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:50.456496 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1436/2872] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:32:50 2022] PUT /identity/v3/projects/d57d10b1337b46afb9c18b949a0f5f3f/users/f15385ec62d34b8fac86a44874e75e8c/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:50.460424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee74d5d-6e19-4dea-b885-9b4fcb9050d8 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:50.470320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee74d5d-6e19-4dea-b885-9b4fcb9050d8 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:50.526438 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c459f60-8d1e-4e04-bb84-b1e2b7183d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:50.527129 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c459f60-8d1e-4e04-bb84-b1e2b7183d07 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:50.527410 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c459f60-8d1e-4e04-bb84-b1e2b7183d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:50.527685 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c459f60-8d1e-4e04-bb84-b1e2b7183d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:50.528317 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c459f60-8d1e-4e04-bb84-b1e2b7183d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:50.529936 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c459f60-8d1e-4e04-bb84-b1e2b7183d07 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:50.540403 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1437/2873] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:32:50.544550 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d2d05-eef3-456b-8734-28519f02154e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:50.555801 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d2d05-eef3-456b-8734-28519f02154e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:50.560491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7f802-7f3e-4035-9e01-25efedbe82f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:50.561170 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7f802-7f3e-4035-9e01-25efedbe82f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:50.561455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7f802-7f3e-4035-9e01-25efedbe82f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:50.561731 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7f802-7f3e-4035-9e01-25efedbe82f0 admin admin] PATH_INFO: `/v3/projects/d57d10b1337b46afb9c18b949a0f5f3f/users/f15385ec62d34b8fac86a44874e75e8c/roles/8ea69cbce665406aa23d4e7e90096e20` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:50.564370 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7f802-7f3e-4035-9e01-25efedbe82f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57d10b1337b46afb9c18b949a0f5f3f, user_id=f15385ec62d34b8fac86a44874e75e8c, role_id=8ea69cbce665406aa23d4e7e90096e20)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:50.567046 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7f802-7f3e-4035-9e01-25efedbe82f0 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:50.577317 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1437/2874] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:32:50 2022] PUT /identity/v3/projects/d57d10b1337b46afb9c18b949a0f5f3f/users/f15385ec62d34b8fac86a44874e75e8c/roles/8ea69cbce665406aa23d4e7e90096e20 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:50.587369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e1d1f-25fa-427a-ac2b-52d4d5d2a446 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:50.587657 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e1d1f-25fa-427a-ac2b-52d4d5d2a446 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:50.587933 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e1d1f-25fa-427a-ac2b-52d4d5d2a446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:50.616328 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-a65e1d1f-25fa-427a-ac2b-52d4d5d2a446 None None] MFA Rules not processed for user `f15385ec62d34b8fac86a44874e75e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:50.617577 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a65e1d1f-25fa-427a-ac2b-52d4d5d2a446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:50.684865 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1438/2875] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:50.696404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7f802-7f3e-4035-9e01-25efedbe82f0 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:50.698186 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f6c7f802-7f3e-4035-9e01-25efedbe82f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:50.725397 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:50.756131 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7f802-7f3e-4035-9e01-25efedbe82f0 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:50.759818 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff03a4-8e6c-41c7-a61a-ed6bf8cf7b26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:50.760408 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff03a4-8e6c-41c7-a61a-ed6bf8cf7b26 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:50.760715 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff03a4-8e6c-41c7-a61a-ed6bf8cf7b26 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:50.760990 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff03a4-8e6c-41c7-a61a-ed6bf8cf7b26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:50.817564 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff03a4-8e6c-41c7-a61a-ed6bf8cf7b26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:50.818937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff03a4-8e6c-41c7-a61a-ed6bf8cf7b26 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:50.827763 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1438/2876] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:32:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:32:52.779969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e1d1f-25fa-427a-ac2b-52d4d5d2a446 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:52.781446 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-a65e1d1f-25fa-427a-ac2b-52d4d5d2a446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:52.794617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e1d1f-25fa-427a-ac2b-52d4d5d2a446 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:52.798173 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb667b5f-4ebd-4f94-8ab0-91b4b1b72754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:52.798770 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb667b5f-4ebd-4f94-8ab0-91b4b1b72754 service cinder] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:52.799043 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb667b5f-4ebd-4f94-8ab0-91b4b1b72754 service cinder] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:52.799314 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb667b5f-4ebd-4f94-8ab0-91b4b1b72754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:52.854877 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb667b5f-4ebd-4f94-8ab0-91b4b1b72754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:52.856332 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb667b5f-4ebd-4f94-8ab0-91b4b1b72754 service cinder] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:52.865207 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1439/2877] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:32:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:32:54.270260 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff03a4-8e6c-41c7-a61a-ed6bf8cf7b26 service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:54.278941 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff03a4-8e6c-41c7-a61a-ed6bf8cf7b26 service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:54.282538 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06576b72-f69c-4e4d-8b3e-5e84dabf13c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:54.283126 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576b72-f69c-4e4d-8b3e-5e84dabf13c5 service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:54.283401 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576b72-f69c-4e4d-8b3e-5e84dabf13c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:54.283681 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576b72-f69c-4e4d-8b3e-5e84dabf13c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:54.293629 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06576b72-f69c-4e4d-8b3e-5e84dabf13c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:54.295248 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06576b72-f69c-4e4d-8b3e-5e84dabf13c5 service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:54.304709 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1439/2878] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:32:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:32:58.818611 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb667b5f-4ebd-4f94-8ab0-91b4b1b72754 service cinder] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:58.831528 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:32:58.880021 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb667b5f-4ebd-4f94-8ab0-91b4b1b72754 service cinder] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:58.884588 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671f4f0-897b-42bf-87bb-f74a749b3814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:58.885231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671f4f0-897b-42bf-87bb-f74a749b3814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:58.885509 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671f4f0-897b-42bf-87bb-f74a749b3814 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:58.885780 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671f4f0-897b-42bf-87bb-f74a749b3814 admin admin] PATH_INFO: `/v3/users/cc917d8dac4248a586dd9fddea086580` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:58.886897 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671f4f0-897b-42bf-87bb-f74a749b3814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc917d8dac4248a586dd9fddea086580)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:58.888823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671f4f0-897b-42bf-87bb-f74a749b3814 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:59.080634 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f671f4f0-897b-42bf-87bb-f74a749b3814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc917d8dac4248a586dd9fddea086580'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:59.091917 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f671f4f0-897b-42bf-87bb-f74a749b3814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc917d8dac4248a586dd9fddea086580'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:59.176802 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f671f4f0-897b-42bf-87bb-f74a749b3814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc917d8dac4248a586dd9fddea086580'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:59.177424 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f671f4f0-897b-42bf-87bb-f74a749b3814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc917d8dac4248a586dd9fddea086580'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:59.189776 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-f671f4f0-897b-42bf-87bb-f74a749b3814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc917d8dac4248a586dd9fddea086580'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:59.195800 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1440/2879] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:32:58 2022] DELETE /identity/v3/users/cc917d8dac4248a586dd9fddea086580 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:32:59.200440 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ae180-c798-4680-956f-b08bee5f7b55 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:59.200946 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ae180-c798-4680-956f-b08bee5f7b55 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:59.201059 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ae180-c798-4680-956f-b08bee5f7b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:59.220845 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-2a4ae180-c798-4680-956f-b08bee5f7b55 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:32:59.222859 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2a4ae180-c798-4680-956f-b08bee5f7b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:59.281362 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1440/2880] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:32:59.291848 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671f4f0-897b-42bf-87bb-f74a749b3814 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:59.293293 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-f671f4f0-897b-42bf-87bb-f74a749b3814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:59.349476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671f4f0-897b-42bf-87bb-f74a749b3814 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:59.353038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e7788-d8bf-45bd-bac0-ee0db27df7b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:59.353619 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e7788-d8bf-45bd-bac0-ee0db27df7b9 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:59.353902 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e7788-d8bf-45bd-bac0-ee0db27df7b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:59.354176 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e7788-d8bf-45bd-bac0-ee0db27df7b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:59.363054 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e7788-d8bf-45bd-bac0-ee0db27df7b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:59.364483 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e7788-d8bf-45bd-bac0-ee0db27df7b9 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:59.373911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1441/2881] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:32:59.771931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ae180-c798-4680-956f-b08bee5f7b55 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:32:59.773382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2a4ae180-c798-4680-956f-b08bee5f7b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:32:59.789376 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ae180-c798-4680-956f-b08bee5f7b55 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:32:59.798168 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:32:59.798168 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:59.798168 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:59.798168 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] PATH_INFO: `/v3/projects/e9d7796441824ab997b675a4ba02663e` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:59.798508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9d7796441824ab997b675a4ba02663e)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:32:59.798508 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:32:59.923133 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9d7796441824ab997b675a4ba02663e', 'request_id': 'req-f199464f-8da4-4abb-9bc1-821239592544'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:59.929624 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9d7796441824ab997b675a4ba02663e', 'request_id': 'req-f199464f-8da4-4abb-9bc1-821239592544'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:32:59.933272 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:00.025265 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9d7796441824ab997b675a4ba02663e', 'request_id': 'req-f199464f-8da4-4abb-9bc1-821239592544'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:00.036973 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] The token cache is being invalidate because project e9d7796441824ab997b675a4ba02663e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:33:00.037261 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:00.037882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:00.039402 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1441/2882] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:32:59 2022] DELETE /identity/v3/projects/e9d7796441824ab997b675a4ba02663e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:00.043542 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e7788-d8bf-45bd-bac0-ee0db27df7b9 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:00.054291 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e7788-d8bf-45bd-bac0-ee0db27df7b9 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:00.087203 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:00.110598 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b68b96-fb3b-4bc9-a872-858c56e9c5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:00.111236 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68b96-fb3b-4bc9-a872-858c56e9c5d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:00.111519 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68b96-fb3b-4bc9-a872-858c56e9c5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:00.111790 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68b96-fb3b-4bc9-a872-858c56e9c5d4 admin admin] PATH_INFO: `/v3/users/49a0f86c9f424d8e9f13362170e65eed` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:00.113470 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b68b96-fb3b-4bc9-a872-858c56e9c5d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49a0f86c9f424d8e9f13362170e65eed)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:00.115843 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b68b96-fb3b-4bc9-a872-858c56e9c5d4 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:00.341713 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-89b68b96-fb3b-4bc9-a872-858c56e9c5d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49a0f86c9f424d8e9f13362170e65eed'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:00.349536 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-89b68b96-fb3b-4bc9-a872-858c56e9c5d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49a0f86c9f424d8e9f13362170e65eed'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:00.427614 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-89b68b96-fb3b-4bc9-a872-858c56e9c5d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49a0f86c9f424d8e9f13362170e65eed'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:00.428256 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-89b68b96-fb3b-4bc9-a872-858c56e9c5d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49a0f86c9f424d8e9f13362170e65eed'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:00.440439 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-89b68b96-fb3b-4bc9-a872-858c56e9c5d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49a0f86c9f424d8e9f13362170e65eed'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:00.446294 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1442/2883] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:33:00 2022] DELETE /identity/v3/users/49a0f86c9f424d8e9f13362170e65eed => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:00.843929 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:00.845450 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:00.859664 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199464f-8da4-4abb-9bc1-821239592544 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:00.915934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:00.916559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:00.916869 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:00.917168 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] PATH_INFO: `/v3/projects/5908ad9b12e94291b50626adec171db1` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:00.918280 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5908ad9b12e94291b50626adec171db1)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:00.920181 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:01.056378 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5908ad9b12e94291b50626adec171db1', 'request_id': 'req-dee983e7-0cf4-45ba-b213-9424f3471e57'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:01.062564 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5908ad9b12e94291b50626adec171db1', 'request_id': 'req-dee983e7-0cf4-45ba-b213-9424f3471e57'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:01.118918 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5908ad9b12e94291b50626adec171db1', 'request_id': 'req-dee983e7-0cf4-45ba-b213-9424f3471e57'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:01.130287 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] The token cache is being invalidate because project 5908ad9b12e94291b50626adec171db1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:33:01.130576 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:01.131198 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:01.132897 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1442/2884] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:33:00 2022] DELETE /identity/v3/projects/5908ad9b12e94291b50626adec171db1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:01.136622 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b68b96-fb3b-4bc9-a872-858c56e9c5d4 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:01.138053 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-89b68b96-fb3b-4bc9-a872-858c56e9c5d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:01.151617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b68b96-fb3b-4bc9-a872-858c56e9c5d4 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:01.208448 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73544b-4738-416a-9817-d6ad16d41f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:01.209132 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73544b-4738-416a-9817-d6ad16d41f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:01.209415 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73544b-4738-416a-9817-d6ad16d41f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:01.209688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73544b-4738-416a-9817-d6ad16d41f8d admin admin] PATH_INFO: `/v3/users/f15385ec62d34b8fac86a44874e75e8c` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:01.210802 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73544b-4738-416a-9817-d6ad16d41f8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f15385ec62d34b8fac86a44874e75e8c)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:01.212866 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73544b-4738-416a-9817-d6ad16d41f8d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:01.395627 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bc73544b-4738-416a-9817-d6ad16d41f8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f15385ec62d34b8fac86a44874e75e8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:01.402465 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bc73544b-4738-416a-9817-d6ad16d41f8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f15385ec62d34b8fac86a44874e75e8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:01.404157 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:01.473031 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bc73544b-4738-416a-9817-d6ad16d41f8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f15385ec62d34b8fac86a44874e75e8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:01.473666 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bc73544b-4738-416a-9817-d6ad16d41f8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f15385ec62d34b8fac86a44874e75e8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:01.485656 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-bc73544b-4738-416a-9817-d6ad16d41f8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f15385ec62d34b8fac86a44874e75e8c'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:01.492001 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1443/2885] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:33:01 2022] DELETE /identity/v3/users/f15385ec62d34b8fac86a44874e75e8c => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:01.909018 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:01.913348 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:01.924584 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee983e7-0cf4-45ba-b213-9424f3471e57 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:01.983143 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22570387-08a5-463f-bc3a-b345ee3d0f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:01.983824 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22570387-08a5-463f-bc3a-b345ee3d0f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:01.984107 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22570387-08a5-463f-bc3a-b345ee3d0f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:01.984382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22570387-08a5-463f-bc3a-b345ee3d0f53 admin admin] PATH_INFO: `/v3/projects/d57d10b1337b46afb9c18b949a0f5f3f` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:01.985518 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22570387-08a5-463f-bc3a-b345ee3d0f53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d57d10b1337b46afb9c18b949a0f5f3f)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:01.987415 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22570387-08a5-463f-bc3a-b345ee3d0f53 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:02.008386 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:02.082187 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-22570387-08a5-463f-bc3a-b345ee3d0f53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd57d10b1337b46afb9c18b949a0f5f3f', 'request_id': 'req-22570387-08a5-463f-bc3a-b345ee3d0f53'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:02.088197 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-22570387-08a5-463f-bc3a-b345ee3d0f53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd57d10b1337b46afb9c18b949a0f5f3f', 'request_id': 'req-22570387-08a5-463f-bc3a-b345ee3d0f53'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:02.167823 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-22570387-08a5-463f-bc3a-b345ee3d0f53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd57d10b1337b46afb9c18b949a0f5f3f', 'request_id': 'req-22570387-08a5-463f-bc3a-b345ee3d0f53'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:02.179155 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-22570387-08a5-463f-bc3a-b345ee3d0f53 admin admin] The token cache is being invalidate because project d57d10b1337b46afb9c18b949a0f5f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:33:02.179439 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-22570387-08a5-463f-bc3a-b345ee3d0f53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:02.180062 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-22570387-08a5-463f-bc3a-b345ee3d0f53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:02.181569 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1443/2886] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:33:01 2022] DELETE /identity/v3/projects/d57d10b1337b46afb9c18b949a0f5f3f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:02.186401 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc29a56-24bb-405a-a8cb-d6d605eca7c5 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:02.186883 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc29a56-24bb-405a-a8cb-d6d605eca7c5 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:02.187145 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc29a56-24bb-405a-a8cb-d6d605eca7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:02.206599 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-ecc29a56-24bb-405a-a8cb-d6d605eca7c5 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:02.208661 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ecc29a56-24bb-405a-a8cb-d6d605eca7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:02.271774 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1444/2887] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:33:02.287316 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ccea3-4fff-48b3-b31b-03d55e6a1b53 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:02.287603 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ccea3-4fff-48b3-b31b-03d55e6a1b53 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:02.287889 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ccea3-4fff-48b3-b31b-03d55e6a1b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:02.307662 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-f29ccea3-4fff-48b3-b31b-03d55e6a1b53 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:02.308985 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-f29ccea3-4fff-48b3-b31b-03d55e6a1b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:02.316931 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1444/2888] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:33:02.322538 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc29a56-24bb-405a-a8cb-d6d605eca7c5 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:02.333013 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc29a56-24bb-405a-a8cb-d6d605eca7c5 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:02.337774 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6afc3-d632-4c25-a7f5-bd02c3e6f107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:02.338369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6afc3-d632-4c25-a7f5-bd02c3e6f107 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:02.338654 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6afc3-d632-4c25-a7f5-bd02c3e6f107 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:02.338954 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6afc3-d632-4c25-a7f5-bd02c3e6f107 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:02.339601 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6afc3-d632-4c25-a7f5-bd02c3e6f107 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:02.339911 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6afc3-d632-4c25-a7f5-bd02c3e6f107 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:02.341540 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6afc3-d632-4c25-a7f5-bd02c3e6f107 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:02.353606 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1445/2889] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:02.357888 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ccea3-4fff-48b3-b31b-03d55e6a1b53 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:02.367512 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ccea3-4fff-48b3-b31b-03d55e6a1b53 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:02.372826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a866f-19f0-4a18-95d7-7960d9875578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:02.373471 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a866f-19f0-4a18-95d7-7960d9875578 admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:02.373755 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a866f-19f0-4a18-95d7-7960d9875578 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:02.374031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a866f-19f0-4a18-95d7-7960d9875578 admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:02.374661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a866f-19f0-4a18-95d7-7960d9875578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:02.376624 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a866f-19f0-4a18-95d7-7960d9875578 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:02.463382 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1445/2890] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:33:02 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:02.467934 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6afc3-d632-4c25-a7f5-bd02c3e6f107 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:02.478662 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6afc3-d632-4c25-a7f5-bd02c3e6f107 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:02.533768 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541de96-fb0b-42ef-8e29-b77c1daefb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:02.534376 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541de96-fb0b-42ef-8e29-b77c1daefb0f admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:02.534668 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541de96-fb0b-42ef-8e29-b77c1daefb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:02.535913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541de96-fb0b-42ef-8e29-b77c1daefb0f admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:02.536904 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541de96-fb0b-42ef-8e29-b77c1daefb0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:02.538852 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541de96-fb0b-42ef-8e29-b77c1daefb0f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:02.628113 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1446/2891] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:33:02 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:33:02.632934 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a866f-19f0-4a18-95d7-7960d9875578 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:02.643306 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a866f-19f0-4a18-95d7-7960d9875578 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:02.648873 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac561e59-62e5-4ad8-a860-83bf1cd0bd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:02.649455 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac561e59-62e5-4ad8-a860-83bf1cd0bd8e admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:02.649741 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac561e59-62e5-4ad8-a860-83bf1cd0bd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:02.650015 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac561e59-62e5-4ad8-a860-83bf1cd0bd8e admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:02.650650 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac561e59-62e5-4ad8-a860-83bf1cd0bd8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:02.652242 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac561e59-62e5-4ad8-a860-83bf1cd0bd8e admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:02.662748 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1446/2892] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:33:02.666702 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541de96-fb0b-42ef-8e29-b77c1daefb0f admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:02.676763 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541de96-fb0b-42ef-8e29-b77c1daefb0f admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:02.681612 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f675084-1b67-434f-a939-632b30b1db0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:02.682146 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f675084-1b67-434f-a939-632b30b1db0f admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:02.682425 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f675084-1b67-434f-a939-632b30b1db0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:02.682698 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f675084-1b67-434f-a939-632b30b1db0f admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:02.683334 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f675084-1b67-434f-a939-632b30b1db0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:02.684969 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f675084-1b67-434f-a939-632b30b1db0f admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:02.696382 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1447/2893] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:33:02.700322 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac561e59-62e5-4ad8-a860-83bf1cd0bd8e admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:02.711045 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac561e59-62e5-4ad8-a860-83bf1cd0bd8e admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:02.715610 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faaa939-f837-4020-8621-d4e58dab97a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:02.716268 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaa939-f837-4020-8621-d4e58dab97a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:02.716547 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaa939-f837-4020-8621-d4e58dab97a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:02.716855 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaa939-f837-4020-8621-d4e58dab97a0 admin admin] PATH_INFO: `/v3/projects/9e4af84710ea488ba5afc7a06448f3d6/users/a9194142fa4849109e60998aa45ff4e2/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:02.732240 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faaa939-f837-4020-8621-d4e58dab97a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e4af84710ea488ba5afc7a06448f3d6, user_id=a9194142fa4849109e60998aa45ff4e2, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:02.734906 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faaa939-f837-4020-8621-d4e58dab97a0 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:02.804294 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1447/2894] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:33:02 2022] PUT /identity/v3/projects/9e4af84710ea488ba5afc7a06448f3d6/users/a9194142fa4849109e60998aa45ff4e2/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:02.814487 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc51b32-4350-4e77-af75-a7a4677d544b None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:02.814771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc51b32-4350-4e77-af75-a7a4677d544b None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:02.815055 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc51b32-4350-4e77-af75-a7a4677d544b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:02.842980 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-2bc51b32-4350-4e77-af75-a7a4677d544b None None] MFA Rules not processed for user `a9194142fa4849109e60998aa45ff4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:02.844235 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-2bc51b32-4350-4e77-af75-a7a4677d544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:02.872900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:02.907804 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1448/2895] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:33:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:33:02.920236 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faaa939-f837-4020-8621-d4e58dab97a0 admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:02.922012 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0faaa939-f837-4020-8621-d4e58dab97a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:02.979437 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faaa939-f837-4020-8621-d4e58dab97a0 admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:02.983124 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4596b1-e2f3-4bf7-9291-e0508c3c88c4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38de805ba5eb4aa58cf95d2082c2d216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:02.983724 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4596b1-e2f3-4bf7-9291-e0508c3c88c4 service glance] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:02.984006 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4596b1-e2f3-4bf7-9291-e0508c3c88c4 service glance] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:02.984279 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4596b1-e2f3-4bf7-9291-e0508c3c88c4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:02.993161 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4596b1-e2f3-4bf7-9291-e0508c3c88c4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:02.994600 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4596b1-e2f3-4bf7-9291-e0508c3c88c4 service glance] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:03.003396 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1448/2896] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:33:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:33:12.999335 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:13.055811 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=14 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:36.953866 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc51b32-4350-4e77-af75-a7a4677d544b None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:37.023632 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc51b32-4350-4e77-af75-a7a4677d544b None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:37.028601 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cc83e-e592-4b6b-b296-a66aac0d80ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:37.029371 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cc83e-e592-4b6b-b296-a66aac0d80ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:37.029771 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cc83e-e592-4b6b-b296-a66aac0d80ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:37.030127 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cc83e-e592-4b6b-b296-a66aac0d80ad admin admin] PATH_INFO: `/v3/users/a9194142fa4849109e60998aa45ff4e2` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:37.031319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51cc83e-e592-4b6b-b296-a66aac0d80ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9194142fa4849109e60998aa45ff4e2)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:37.033354 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51cc83e-e592-4b6b-b296-a66aac0d80ad admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:37.197931 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b51cc83e-e592-4b6b-b296-a66aac0d80ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9194142fa4849109e60998aa45ff4e2'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:37.204541 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b51cc83e-e592-4b6b-b296-a66aac0d80ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9194142fa4849109e60998aa45ff4e2'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:37.264269 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b51cc83e-e592-4b6b-b296-a66aac0d80ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9194142fa4849109e60998aa45ff4e2'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:37.264928 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b51cc83e-e592-4b6b-b296-a66aac0d80ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9194142fa4849109e60998aa45ff4e2'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:37.277369 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-b51cc83e-e592-4b6b-b296-a66aac0d80ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9194142fa4849109e60998aa45ff4e2'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:37.283290 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1449/2897] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:33:36 2022] DELETE /identity/v3/users/a9194142fa4849109e60998aa45ff4e2 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:33:37.288938 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f435117-e752-469a-9209-5a39483a9115 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:37.289426 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f435117-e752-469a-9209-5a39483a9115 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:37.289521 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f435117-e752-469a-9209-5a39483a9115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:37.308661 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-0f435117-e752-469a-9209-5a39483a9115 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:37.312817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0f435117-e752-469a-9209-5a39483a9115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:37.367798 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1449/2898] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:33:37.377423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cc83e-e592-4b6b-b296-a66aac0d80ad admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:37.378850 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-b51cc83e-e592-4b6b-b296-a66aac0d80ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:37.435773 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cc83e-e592-4b6b-b296-a66aac0d80ad admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:37.439421 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2026e-6290-4b45-8252-47c4ab342ee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:37.440020 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2026e-6290-4b45-8252-47c4ab342ee6 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:37.440304 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2026e-6290-4b45-8252-47c4ab342ee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:37.440580 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2026e-6290-4b45-8252-47c4ab342ee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:37.449497 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2026e-6290-4b45-8252-47c4ab342ee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:37.450869 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2026e-6290-4b45-8252-47c4ab342ee6 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:37.460879 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1450/2899] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:33:37.867522 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f435117-e752-469a-9209-5a39483a9115 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:37.869087 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-0f435117-e752-469a-9209-5a39483a9115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:37.881604 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f435117-e752-469a-9209-5a39483a9115 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:37.886418 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:37.887901 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:37.888247 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592 admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:37.888559 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592 admin admin] PATH_INFO: `/v3/projects/9e4af84710ea488ba5afc7a06448f3d6` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:37.889891 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e4af84710ea488ba5afc7a06448f3d6)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:37.891851 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592 admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:37.974735 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e4af84710ea488ba5afc7a06448f3d6', 'request_id': 'req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:37.980895 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e4af84710ea488ba5afc7a06448f3d6', 'request_id': 'req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:38.070079 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e4af84710ea488ba5afc7a06448f3d6', 'request_id': 'req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:38.080618 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592 admin admin] The token cache is being invalidate because project 9e4af84710ea488ba5afc7a06448f3d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:33:38.080935 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:38.081594 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-1c6a10bb-49ee-4cb2-ab9c-0a9191a96592 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:33:38.083068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1450/2900] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:33:37 2022] DELETE /identity/v3/projects/9e4af84710ea488ba5afc7a06448f3d6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:38.088752 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79169ed-7b2a-42e1-bcaf-0a2784659ca6 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.091308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79169ed-7b2a-42e1-bcaf-0a2784659ca6 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.091308 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79169ed-7b2a-42e1-bcaf-0a2784659ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.109038 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-d79169ed-7b2a-42e1-bcaf-0a2784659ca6 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:38.110231 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-d79169ed-7b2a-42e1-bcaf-0a2784659ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:38.127039 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:38.168837 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1451/2901] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:33:38.176781 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c087f-ba43-4e6a-b4d7-e6c06a57a4ea None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.177560 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c087f-ba43-4e6a-b4d7-e6c06a57a4ea None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.178068 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c087f-ba43-4e6a-b4d7-e6c06a57a4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.198242 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-cd1c087f-ba43-4e6a-b4d7-e6c06a57a4ea None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:38.199432 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-cd1c087f-ba43-4e6a-b4d7-e6c06a57a4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:38.207236 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1451/2902] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:33:38.211704 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79169ed-7b2a-42e1-bcaf-0a2784659ca6 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:38.221630 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79169ed-7b2a-42e1-bcaf-0a2784659ca6 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:38.226423 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2796bc0-0db1-41b0-bf18-0ad77060cc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:38.226991 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2796bc0-0db1-41b0-bf18-0ad77060cc27 admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.227300 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2796bc0-0db1-41b0-bf18-0ad77060cc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.227576 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2796bc0-0db1-41b0-bf18-0ad77060cc27 admin admin] PATH_INFO: `/v3/domains` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.228221 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2796bc0-0db1-41b0-bf18-0ad77060cc27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:38.228554 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2796bc0-0db1-41b0-bf18-0ad77060cc27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:38.230182 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2796bc0-0db1-41b0-bf18-0ad77060cc27 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:38.241956 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1452/2903] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 24 08:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:38.246150 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c087f-ba43-4e6a-b4d7-e6c06a57a4ea None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:38.255745 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c087f-ba43-4e6a-b4d7-e6c06a57a4ea None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:38.260404 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df3213-4fab-4fdf-ae55-f5242142f75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:38.261056 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df3213-4fab-4fdf-ae55-f5242142f75f admin admin] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.261337 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df3213-4fab-4fdf-ae55-f5242142f75f admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.261635 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df3213-4fab-4fdf-ae55-f5242142f75f admin admin] PATH_INFO: `/v3/projects` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.262221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df3213-4fab-4fdf-ae55-f5242142f75f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:38.264105 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df3213-4fab-4fdf-ae55-f5242142f75f admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:38.346064 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1452/2904] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 24 08:33:38 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:38.350414 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2796bc0-0db1-41b0-bf18-0ad77060cc27 admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:38.360336 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2796bc0-0db1-41b0-bf18-0ad77060cc27 admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:38.417870 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351fec9-f73d-4112-a035-9924f427b02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:38.418474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351fec9-f73d-4112-a035-9924f427b02b admin admin] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.418756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351fec9-f73d-4112-a035-9924f427b02b admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.419032 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351fec9-f73d-4112-a035-9924f427b02b admin admin] PATH_INFO: `/v3/users` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.419652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351fec9-f73d-4112-a035-9924f427b02b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:38.421620 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351fec9-f73d-4112-a035-9924f427b02b admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:38.498823 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1453/2905] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 24 08:33:38 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:38.502835 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df3213-4fab-4fdf-ae55-f5242142f75f admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:38.514673 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df3213-4fab-4fdf-ae55-f5242142f75f admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:38.519324 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e748ae5-81f1-4b34-95ff-4a4051fb18ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:38.519961 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e748ae5-81f1-4b34-95ff-4a4051fb18ee admin admin] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.520244 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e748ae5-81f1-4b34-95ff-4a4051fb18ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.520517 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e748ae5-81f1-4b34-95ff-4a4051fb18ee admin admin] PATH_INFO: `/v3/roles` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.521221 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e748ae5-81f1-4b34-95ff-4a4051fb18ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:38.522826 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e748ae5-81f1-4b34-95ff-4a4051fb18ee admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:38.533307 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1453/2906] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:33:38.537819 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351fec9-f73d-4112-a035-9924f427b02b admin admin] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:38.547583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351fec9-f73d-4112-a035-9924f427b02b admin admin] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:38.552910 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aba107-84a1-4a4c-b0ae-194d23732d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:38.553494 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aba107-84a1-4a4c-b0ae-194d23732d6d admin admin] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.553772 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aba107-84a1-4a4c-b0ae-194d23732d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.554065 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aba107-84a1-4a4c-b0ae-194d23732d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.554699 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aba107-84a1-4a4c-b0ae-194d23732d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:38.556266 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aba107-84a1-4a4c-b0ae-194d23732d6d admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:38.566652 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1454/2907] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 24 08:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:33:38.570980 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e748ae5-81f1-4b34-95ff-4a4051fb18ee admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:38.580669 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e748ae5-81f1-4b34-95ff-4a4051fb18ee admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:38.585571 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d2581-a3cf-4d59-9a78-c7d008d25f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:38.586206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d2581-a3cf-4d59-9a78-c7d008d25f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.586486 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d2581-a3cf-4d59-9a78-c7d008d25f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.586755 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d2581-a3cf-4d59-9a78-c7d008d25f4b admin admin] PATH_INFO: `/v3/projects/bee8c1b48a704a41a6768092e0792177/users/188bf9f182844a138d8187428a12d9c4/roles/9fe465cfe8764112a4cc92f3480326b8` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.601848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d2581-a3cf-4d59-9a78-c7d008d25f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bee8c1b48a704a41a6768092e0792177, user_id=188bf9f182844a138d8187428a12d9c4, role_id=9fe465cfe8764112a4cc92f3480326b8)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:38.604491 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d2581-a3cf-4d59-9a78-c7d008d25f4b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:38.692958 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1454/2908] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 24 08:33:38 2022] PUT /identity/v3/projects/bee8c1b48a704a41a6768092e0792177/users/188bf9f182844a138d8187428a12d9c4/roles/9fe465cfe8764112a4cc92f3480326b8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:38.704205 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576f687-bdf0-4f81-92f1-a5484d9a1ee1 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.704493 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576f687-bdf0-4f81-92f1-a5484d9a1ee1 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.704796 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576f687-bdf0-4f81-92f1-a5484d9a1ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.732165 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-3576f687-bdf0-4f81-92f1-a5484d9a1ee1 None None] MFA Rules not processed for user `188bf9f182844a138d8187428a12d9c4`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:38.733396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3576f687-bdf0-4f81-92f1-a5484d9a1ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:38.797377 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1455/2909] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:33:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:33:38.808951 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d2581-a3cf-4d59-9a78-c7d008d25f4b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:38.810644 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-8c5d2581-a3cf-4d59-9a78-c7d008d25f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:38.832125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:38.876968 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d2581-a3cf-4d59-9a78-c7d008d25f4b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:38.880900 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd1b15-cabb-41e3-b2f6-7fb8e308d5dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e57131a5be4af785166bc03a3337a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:38.881480 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd1b15-cabb-41e3-b2f6-7fb8e308d5dd service cinder] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.881759 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd1b15-cabb-41e3-b2f6-7fb8e308d5dd service cinder] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.882033 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd1b15-cabb-41e3-b2f6-7fb8e308d5dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.890052 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd1b15-cabb-41e3-b2f6-7fb8e308d5dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:38.891475 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd1b15-cabb-41e3-b2f6-7fb8e308d5dd service cinder] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:38.900921 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1455/2910] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:33:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:33:40.544080 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36c53e-b967-433f-8796-ec289a306800 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:40.544523 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36c53e-b967-433f-8796-ec289a306800 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:40.544616 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36c53e-b967-433f-8796-ec289a306800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:40.563125 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-ba36c53e-b967-433f-8796-ec289a306800 None None] MFA Rules not processed for user `188bf9f182844a138d8187428a12d9c4`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:40.564468 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ba36c53e-b967-433f-8796-ec289a306800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:40.571319 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1456/2911] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:33:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:33:42.178461 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd1b15-cabb-41e3-b2f6-7fb8e308d5dd service cinder] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:42.179866 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-97fd1b15-cabb-41e3-b2f6-7fb8e308d5dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:42.251872 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd1b15-cabb-41e3-b2f6-7fb8e308d5dd service cinder] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:42.257889 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666d9f6-b10a-4269-b69a-673071ebcaab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:42.258517 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d9f6-b10a-4269-b69a-673071ebcaab service placement] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:42.258797 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d9f6-b10a-4269-b69a-673071ebcaab service placement] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:42.259072 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d9f6-b10a-4269-b69a-673071ebcaab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:42.260848 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-3666d9f6-b10a-4269-b69a-673071ebcaab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:42.272819 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666d9f6-b10a-4269-b69a-673071ebcaab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:42.274390 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666d9f6-b10a-4269-b69a-673071ebcaab service placement] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:42.284411 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1456/2912] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:33:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:33:45.699672 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36c53e-b967-433f-8796-ec289a306800 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.701205 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-ba36c53e-b967-433f-8796-ec289a306800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:45.714658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:45.760339 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36c53e-b967-433f-8796-ec289a306800 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.763987 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020413e5-c3f5-4fdc-a110-6b45d37195c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.764593 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020413e5-c3f5-4fdc-a110-6b45d37195c9 service neutron] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.764900 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020413e5-c3f5-4fdc-a110-6b45d37195c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.765171 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020413e5-c3f5-4fdc-a110-6b45d37195c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:45.766913 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-020413e5-c3f5-4fdc-a110-6b45d37195c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:45.776720 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020413e5-c3f5-4fdc-a110-6b45d37195c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:45.778168 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020413e5-c3f5-4fdc-a110-6b45d37195c9 service neutron] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:45.787166 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1457/2913] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:33:46.508416 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666d9f6-b10a-4269-b69a-673071ebcaab service placement] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:46.509937 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-3666d9f6-b10a-4269-b69a-673071ebcaab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:46.588511 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666d9f6-b10a-4269-b69a-673071ebcaab service placement] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:46.594070 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b811777-24f8-40f6-9b72-d6d5156a10a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:46.594742 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b811777-24f8-40f6-9b72-d6d5156a10a2 service nova] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:46.595031 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b811777-24f8-40f6-9b72-d6d5156a10a2 service nova] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:46.595300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b811777-24f8-40f6-9b72-d6d5156a10a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:46.597147 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-1b811777-24f8-40f6-9b72-d6d5156a10a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:46.609957 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b811777-24f8-40f6-9b72-d6d5156a10a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:46.611593 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b811777-24f8-40f6-9b72-d6d5156a10a2 service nova] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:46.623730 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1457/2914] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:33:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:33:50.742906 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020413e5-c3f5-4fdc-a110-6b45d37195c9 service neutron] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:50.744490 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-020413e5-c3f5-4fdc-a110-6b45d37195c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:50.760681 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020413e5-c3f5-4fdc-a110-6b45d37195c9 service neutron] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:50.766383 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965c90b-0174-413f-bb84-bb41a6f863c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdf81f09006a42299c0807f77f16c748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:50.766922 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965c90b-0174-413f-bb84-bb41a6f863c5 service nova] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:50.767285 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965c90b-0174-413f-bb84-bb41a6f863c5 service nova] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:50.767549 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965c90b-0174-413f-bb84-bb41a6f863c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:50.769251 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3965c90b-0174-413f-bb84-bb41a6f863c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:50.781370 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965c90b-0174-413f-bb84-bb41a6f863c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:50.782946 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965c90b-0174-413f-bb84-bb41a6f863c5 service nova] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:50.794320 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1458/2915] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:33:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:33:56.619803 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=87 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:56.813778 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565b941-5a0f-430d-b9a6-3b1cf4d4d5bc None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:56.814067 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565b941-5a0f-430d-b9a6-3b1cf4d4d5bc None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:56.814380 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565b941-5a0f-430d-b9a6-3b1cf4d4d5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:56.833554 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-6565b941-5a0f-430d-b9a6-3b1cf4d4d5bc None None] MFA Rules not processed for user `be379e1a6add42b69e9fdd676efca280`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:56.835313 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-6565b941-5a0f-430d-b9a6-3b1cf4d4d5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:56.890854 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1458/2916] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 24 08:33:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:33:57.001220 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965c90b-0174-413f-bb84-bb41a6f863c5 service nova] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:57.002607 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-3965c90b-0174-413f-bb84-bb41a6f863c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:57.069511 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965c90b-0174-413f-bb84-bb41a6f863c5 service nova] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:57.074594 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d778e-3d5d-4ea6-96bb-b64e08156e5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b629ad3e6b9f4723b103f9845df2aaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:57.075192 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d778e-3d5d-4ea6-96bb-b64e08156e5d service swift] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:57.075474 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d778e-3d5d-4ea6-96bb-b64e08156e5d service swift] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:57.075746 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d778e-3d5d-4ea6-96bb-b64e08156e5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:57.083558 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d778e-3d5d-4ea6-96bb-b64e08156e5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:57.085930 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d778e-3d5d-4ea6-96bb-b64e08156e5d service swift] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:57.093761 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1459/2917] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 24 08:33:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:33:57.856392 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd0303-6c25-453c-9ccf-6fa09256ffe1 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:57.856855 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd0303-6c25-453c-9ccf-6fa09256ffe1 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:57.856990 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd0303-6c25-453c-9ccf-6fa09256ffe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:57.876081 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-4dcd0303-6c25-453c-9ccf-6fa09256ffe1 None None] MFA Rules not processed for user `be379e1a6add42b69e9fdd676efca280`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:57.877318 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-4dcd0303-6c25-453c-9ccf-6fa09256ffe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:57.885010 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1459/2918] 10.222.0.26 () {58 vars in 967 bytes} [Wed Aug 24 08:33:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:33:57.896842 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d778e-3d5d-4ea6-96bb-b64e08156e5d service swift] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:57.907496 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d778e-3d5d-4ea6-96bb-b64e08156e5d service swift] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:57.913518 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84481c-dab1-43e6-aac9-1e3150ac30d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b629ad3e6b9f4723b103f9845df2aaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:57.914123 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84481c-dab1-43e6-aac9-1e3150ac30d8 service swift] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:57.914408 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84481c-dab1-43e6-aac9-1e3150ac30d8 service swift] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:57.914686 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84481c-dab1-43e6-aac9-1e3150ac30d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:57.922703 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84481c-dab1-43e6-aac9-1e3150ac30d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:57.924271 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84481c-dab1-43e6-aac9-1e3150ac30d8 service swift] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:57.932754 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1460/2919] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 24 08:33:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:33:58.549553 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a495e-d52a-4276-832b-73e17e39d714 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:58.549840 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a495e-d52a-4276-832b-73e17e39d714 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:58.550116 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a495e-d52a-4276-832b-73e17e39d714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:58.569607 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-465a495e-d52a-4276-832b-73e17e39d714 None None] MFA Rules not processed for user `be379e1a6add42b69e9fdd676efca280`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:58.570814 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-465a495e-d52a-4276-832b-73e17e39d714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:58.578008 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1460/2920] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 24 08:33:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:33:58.591688 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84481c-dab1-43e6-aac9-1e3150ac30d8 service swift] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:58.603057 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84481c-dab1-43e6-aac9-1e3150ac30d8 service swift] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:58.608019 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165060f-b9c5-4aaa-b01b-da19351d1270 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b629ad3e6b9f4723b103f9845df2aaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:58.608617 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165060f-b9c5-4aaa-b01b-da19351d1270 service swift] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:58.608987 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165060f-b9c5-4aaa-b01b-da19351d1270 service swift] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:58.609271 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165060f-b9c5-4aaa-b01b-da19351d1270 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:58.617626 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165060f-b9c5-4aaa-b01b-da19351d1270 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:58.619215 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165060f-b9c5-4aaa-b01b-da19351d1270 service swift] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:58.627181 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1461/2921] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 24 08:33:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:33:59.225243 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40665a-e495-4c94-be87-291ff5441bc9 None None] REQUEST_METHOD: `POST` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:59.225726 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40665a-e495-4c94-be87-291ff5441bc9 None None] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:59.225814 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40665a-e495-4c94-be87-291ff5441bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:59.244685 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.auth.core [None req-7b40665a-e495-4c94-be87-291ff5441bc9 None None] MFA Rules not processed for user `be379e1a6add42b69e9fdd676efca280`. Rule list: `[]` (Enabled: `True`). {{(pid=88250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:59.245882 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-7b40665a-e495-4c94-be87-291ff5441bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:59.252910 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1461/2922] 10.222.0.26 () {58 vars in 967 bytes} [Wed Aug 24 08:33:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:33:59.265309 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165060f-b9c5-4aaa-b01b-da19351d1270 service swift] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:59.275938 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165060f-b9c5-4aaa-b01b-da19351d1270 service swift] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:59.281062 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d9e03-115d-477a-ba6d-d7729407077b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b629ad3e6b9f4723b103f9845df2aaaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:59.281658 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d9e03-115d-477a-ba6d-d7729407077b service swift] REQUEST_METHOD: `GET` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:59.281934 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d9e03-115d-477a-ba6d-d7729407077b service swift] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:59.282206 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d9e03-115d-477a-ba6d-d7729407077b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:59.290398 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d9e03-115d-477a-ba6d-d7729407077b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:59.291948 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d9e03-115d-477a-ba6d-d7729407077b service swift] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:59.299717 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1462/2923] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 24 08:33:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:34:01.918162 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40665a-e495-4c94-be87-291ff5441bc9 None None] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:01.979389 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40665a-e495-4c94-be87-291ff5441bc9 None None] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:01.983878 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd81224-a565-48cd-933f-4bd51f97d19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:01.984507 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd81224-a565-48cd-933f-4bd51f97d19b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:01.984817 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd81224-a565-48cd-933f-4bd51f97d19b admin admin] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:01.985120 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd81224-a565-48cd-933f-4bd51f97d19b admin admin] PATH_INFO: `/v3/users/188bf9f182844a138d8187428a12d9c4` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:01.986250 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd81224-a565-48cd-933f-4bd51f97d19b admin admin] RBAC: Authorizing `identity:delete_user(user_id=188bf9f182844a138d8187428a12d9c4)` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:01.988171 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd81224-a565-48cd-933f-4bd51f97d19b admin admin] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:02.176162 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8bd81224-a565-48cd-933f-4bd51f97d19b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '188bf9f182844a138d8187428a12d9c4'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:34:02.248374 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8bd81224-a565-48cd-933f-4bd51f97d19b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '188bf9f182844a138d8187428a12d9c4'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:34:02.249000 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8bd81224-a565-48cd-933f-4bd51f97d19b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '188bf9f182844a138d8187428a12d9c4'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:34:02.256125 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8bd81224-a565-48cd-933f-4bd51f97d19b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '188bf9f182844a138d8187428a12d9c4'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:34:02.268551 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.notifications [None req-8bd81224-a565-48cd-933f-4bd51f97d19b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '188bf9f182844a138d8187428a12d9c4'} {{(pid=88250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:34:02.275151 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1462/2924] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 24 08:34:01 2022] DELETE /identity/v3/users/188bf9f182844a138d8187428a12d9c4 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:02.279583 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d25e80-f27f-4d3c-ae6e-b1e1be381817 None None] REQUEST_METHOD: `POST` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:02.280035 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d25e80-f27f-4d3c-ae6e-b1e1be381817 None None] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:02.280161 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d25e80-f27f-4d3c-ae6e-b1e1be381817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:02.298923 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.auth.core [None req-88d25e80-f27f-4d3c-ae6e-b1e1be381817 None None] MFA Rules not processed for user `7275460c8c2e486f8768d1910d8aaa60`. Rule list: `[]` (Enabled: `True`). {{(pid=88249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:02.300111 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-88d25e80-f27f-4d3c-ae6e-b1e1be381817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:02.343396 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=100 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:02.357344 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1463/2925] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 24 08:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:34:02.367206 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd81224-a565-48cd-933f-4bd51f97d19b admin admin] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:02.368702 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-8bd81224-a565-48cd-933f-4bd51f97d19b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:02.376245 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:02.427952 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd81224-a565-48cd-933f-4bd51f97d19b admin admin] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:02.432300 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14f07b-5654-4439-80bc-60a3da6f739f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e61402c01c4f9ba5f18e94960bc313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:02.432951 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14f07b-5654-4439-80bc-60a3da6f739f service neutron] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:02.433224 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14f07b-5654-4439-80bc-60a3da6f739f service neutron] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:02.433519 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14f07b-5654-4439-80bc-60a3da6f739f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:02.442991 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14f07b-5654-4439-80bc-60a3da6f739f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:02.445071 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14f07b-5654-4439-80bc-60a3da6f739f service neutron] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:02.454665 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1463/2926] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:34:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:34:02.696397 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d25e80-f27f-4d3c-ae6e-b1e1be381817 None None] Authenticating user token {{(pid=88249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:02.697884 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.fernet_utils [None req-88d25e80-f27f-4d3c-ae6e-b1e1be381817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:02.709729 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d25e80-f27f-4d3c-ae6e-b1e1be381817 None None] Validating token access rules against request {{(pid=88249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:02.715060 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab374d6-6be6-40d4-90cc-a91936dbc342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7275460c8c2e486f8768d1910d8aaa60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12ef7b65bdcb4d6d980c3b5faea49511', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:02.715778 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab374d6-6be6-40d4-90cc-a91936dbc342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:02.716071 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab374d6-6be6-40d4-90cc-a91936dbc342 admin admin] SCRIPT_NAME: `/identity` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:02.716348 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab374d6-6be6-40d4-90cc-a91936dbc342 admin admin] PATH_INFO: `/v3/projects/bee8c1b48a704a41a6768092e0792177` {{(pid=88249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:02.717557 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab374d6-6be6-40d4-90cc-a91936dbc342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bee8c1b48a704a41a6768092e0792177)` {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:02.719457 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab374d6-6be6-40d4-90cc-a91936dbc342 admin admin] RBAC: Authorization granted {{(pid=88249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:02.848757 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1ab374d6-6be6-40d4-90cc-a91936dbc342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bee8c1b48a704a41a6768092e0792177', 'request_id': 'req-1ab374d6-6be6-40d4-90cc-a91936dbc342'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:34:02.855756 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1ab374d6-6be6-40d4-90cc-a91936dbc342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bee8c1b48a704a41a6768092e0792177', 'request_id': 'req-1ab374d6-6be6-40d4-90cc-a91936dbc342'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:34:02.861581 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1ab374d6-6be6-40d4-90cc-a91936dbc342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bee8c1b48a704a41a6768092e0792177', 'request_id': 'req-1ab374d6-6be6-40d4-90cc-a91936dbc342'} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:34:02.943565 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1ab374d6-6be6-40d4-90cc-a91936dbc342 admin admin] The token cache is being invalidate because project bee8c1b48a704a41a6768092e0792177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:34:02.943781 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1ab374d6-6be6-40d4-90cc-a91936dbc342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:34:02.944476 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG keystone.notifications [None req-1ab374d6-6be6-40d4-90cc-a91936dbc342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:34:02.946267 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: [pid: 88249|app: 0|req: 1464/2927] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 24 08:34:02 2022] DELETE /identity/v3/projects/bee8c1b48a704a41a6768092e0792177 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:12.589423 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=24 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:12.940230 dv-cinder-854274-1-iscsi devstack@keystone.service[88249]: DEBUG dbcounter [-] [88249] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:46.390940 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14f07b-5654-4439-80bc-60a3da6f739f service neutron] Authenticating user token {{(pid=88250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:46.392346 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-2a14f07b-5654-4439-80bc-60a3da6f739f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:46.461238 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14f07b-5654-4439-80bc-60a3da6f739f service neutron] Validating token access rules against request {{(pid=88250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:46.519441 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472f2a9-0627-4093-a2b8-ae6e8c5b2b38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea3816d5ae054741ac2c5a2eec3eb617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc060764023e44a7ade80eeed98a235f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:46.520033 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472f2a9-0627-4093-a2b8-ae6e8c5b2b38 service placement] REQUEST_METHOD: `GET` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:46.520315 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472f2a9-0627-4093-a2b8-ae6e8c5b2b38 service placement] SCRIPT_NAME: `/identity` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:46.520589 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472f2a9-0627-4093-a2b8-ae6e8c5b2b38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:46.522314 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.fernet_utils [None req-7472f2a9-0627-4093-a2b8-ae6e8c5b2b38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:46.534658 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472f2a9-0627-4093-a2b8-ae6e8c5b2b38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:46.536253 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472f2a9-0627-4093-a2b8-ae6e8c5b2b38 service placement] RBAC: Authorization granted {{(pid=88250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:46.546984 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: [pid: 88250|app: 0|req: 1464/2928] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 24 08:35:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:35:56.542637 dv-cinder-854274-1-iscsi devstack@keystone.service[88250]: DEBUG dbcounter [-] [88250] Writing DB stats keystone:SELECT=58 {{(pid=88250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}